-- Logs begin at Fri 2022-07-29 12:07:38 UTC, end at Fri 2022-07-29 13:38:05 UTC. -- Jul 29 12:36:02.139934 dv-cinder-819149-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 29 12:36:02.149179 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 29 12:36:02 2022] *** Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: nodename: dv-cinder-819149-16-iscsi Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: machine: x86_64 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: clock source: unix Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: pcre jit disabled Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: detected number of CPU cores: 4 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: current working directory: / Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: detected binary path: /usr/bin/uwsgi-core Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: your processes number limit is 47680 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: your memory page size is 4096 bytes Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: detected max file descriptor number: 2048 Jul 29 12:36:02.151261 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: lock engine: pthread robust mutexes Jul 29 12:36:02.151999 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: thunder lock: enabled Jul 29 12:36:02.151999 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 29 12:36:02.151999 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 29 12:36:02.190733 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: Python main interpreter initialized at 0x55e8051ec9c0 Jul 29 12:36:02.190733 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: python threads support enabled Jul 29 12:36:02.190733 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: your server socket listen backlog is limited to 100 connections Jul 29 12:36:02.190733 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: your mercy for graceful operations on workers is 90 seconds Jul 29 12:36:02.191050 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: mapped 403077 bytes (393 KB) for 2 cores Jul 29 12:36:02.191146 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: *** Operational MODE: preforking *** Jul 29 12:36:02.191422 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: *** uWSGI is running in multiple interpreter mode *** Jul 29 12:36:02.191422 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: spawned uWSGI master process (pid: 91810) Jul 29 12:36:02.191620 dv-cinder-819149-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 29 12:36:02.192372 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: spawned uWSGI worker 1 (pid: 91819, cores: 1) Jul 29 12:36:02.192988 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: spawned uWSGI worker 2 (pid: 91820, cores: 1) Jul 29 12:36:02.192988 dv-cinder-819149-16-iscsi devstack@keystone.service[91810]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 29 12:36:07.733406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 12:36:07.733406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: from cryptography.utils import int_from_bytes Jul 29 12:36:07.734274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 12:36:07.734274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: from cryptography.utils import int_from_bytes Jul 29 12:36:07.783658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 12:36:07.783893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 12:36:07.784091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] command line args: [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 12:36:07.784405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 12:36:07.784587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ================================================================================ {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 12:36:07.784879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] admin_token = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.785136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] config_dir = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.785329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.785513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] config_source = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.785695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] debug = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.785923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.786405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.786499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.786703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.786953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.787186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.787366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_config_append = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.787545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.787723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_dir = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.787899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_file = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.788153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.788381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.788626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.788871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.789048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.789139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 12:36:07.789139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: from cryptography.utils import int_from_bytes Jul 29 12:36:07.789616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.789695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 12:36:07.789695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: from cryptography.utils import int_from_bytes Jul 29 12:36:07.789892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.789892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.790091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.790214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.790374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.790606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.790852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.791103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.791378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.791874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.792055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] publish_errors = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.792289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.792517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.792696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.792939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.793171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.793403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] standard_threads = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.793631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.793897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.794076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.794252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] use_journal = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.794427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] use_json = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.794602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] use_stderr = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.794775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] use_syslog = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.795004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.795255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.795492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.795751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.795990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.796237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.796474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.796719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.796954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.797200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.797444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.797693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.797975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.798769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.799004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.799250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.799496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.799732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.799985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.800238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.800481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.800715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.800969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.801230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.801480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.801726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.801984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.802217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.802446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.802680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.802927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.803172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.803400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.803630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.803875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.804119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.804355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.804601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.804832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.805065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.805299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.805528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.805758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.806015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.806245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.806481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.806728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.806959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.807197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.807426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.807660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.807894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.808141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.808370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.808607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.808844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.809078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.809313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.809547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.809777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.810049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.810283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.810520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.810752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.811014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.811250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.811527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.811770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.812045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.812317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.812555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.812804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.813048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.813282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.813516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.813775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.814045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.814284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.814522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.814766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.815003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.815241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.815475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.815709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.815943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.816207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.816448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.816690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.816927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.817168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.817406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.817641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.817900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.818133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.818362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.818593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.818844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.819056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.819297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.819532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.819778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.820016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.820269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.820527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.820769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.821006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.external = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.821239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.821532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.821765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.822020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.password = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.822253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] auth.token = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.822494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.822728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.822975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.823208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.823456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.823702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.823934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.824169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.824415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.824662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.824910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.825148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.825394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.825628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.825914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.826169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.826446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.826701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.826983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.827228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.827493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.827737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.827993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.828229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.828467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.828713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.828959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.829222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.829473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.829712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.829995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.830235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.830489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.830727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.830989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.831225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.831474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.831722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.831959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.832195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.832440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.832675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.832913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.833162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.833412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.833645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.833907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.834131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.834374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.834619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.834865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.835119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.835364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.835631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.835871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.836109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.836340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.836576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.836805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.837039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.837168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 12:36:07.837274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.837361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 12:36:07.837506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.837595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] command line args: [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 12:36:07.837753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.837867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 12:36:07.837997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ================================================================================ {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 12:36:07.838077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.838242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.838362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] admin_token = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.838488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.838576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] config_dir = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.838683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.838761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.838870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] config_source = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.838949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.839056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] debug = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.839185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.839266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.839417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.839499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.839654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.839743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.839885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.839964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.840122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.840201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.840358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.840446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.840551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_config_append = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.840628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.840732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.840811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.840930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_dir = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.841059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.841147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_file = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.841308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.841387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.841556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.841634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.841786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.841898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.842117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.842398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.842643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.842892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.842980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.843130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.843209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.843365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.843444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.843600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.843688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.843835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.843915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.844084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.844162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.844333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.844421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.844591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.844670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.845356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.845688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.845766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.846158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] publish_errors = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.846310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.846470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.846651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.846770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.846891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.847028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.847183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.847263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.847489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.847568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.847741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.847917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] standard_threads = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.847996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.848223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.848301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.848591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.848681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.848843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.848923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.849078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.849156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] use_journal = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.849328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.849414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] use_json = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.849569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.849666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] use_stderr = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.849810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.849966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] use_syslog = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.850068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.850306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.850396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 12:36:07.850551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.850713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.850871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.851954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.852870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.853861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.854924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.855785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] role.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] role.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.856988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.857933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.858930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.859921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.860199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.860311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.860602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.861972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.862862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.863867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.864918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.865954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.866955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.867887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.868844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.connection = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.869986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.870870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.871778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872506 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.872942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.873948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.874164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.874295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.874428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91819) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 12:36:07.874590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.874833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.875078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.875321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.875591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.875834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.876088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.876341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.876584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.876861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.877105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.877360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.877611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.877875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.878118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.878360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.878613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.878857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.879098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.879347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.879587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.879828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.880065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.880309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.880549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.880816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.881063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.881471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.881933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.882334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.882628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.882969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.883328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.883681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.884037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.884472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.884841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.885071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.885319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.885557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.885813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.886080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.886341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.886606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.886852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.887091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.external = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.887327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.887627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.887867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.888102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.password = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.888338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] auth.token = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.888584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.888826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.889079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.889318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.889572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.889856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.890100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.890347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.890599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.890854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.891107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.891350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.891601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.891868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.892142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.892400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.892643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.892899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.893187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.893438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.893707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.893979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.894242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.894480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.894724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.894974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.895225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.895461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.895713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.895954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.896210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.896734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.897000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.897244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.897494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.897734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.898010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.898264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.898505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.898746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.898997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.899233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.899474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.899728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.899981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.900218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.900460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.900695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.900942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.901192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.901443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.901701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.901974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.902249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.902491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.902733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.902969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.903211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.903445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.903684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.903813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.903947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.904078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.904217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.904345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.904487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.904620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.904871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.905122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.905360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.905609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.905878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.906117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.906357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.906590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.906830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.907069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.907834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.908097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.908340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.908594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.908846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.909082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.909317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.909569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.909847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.910081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.910326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.910567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.910802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.911047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.911310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.911546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.911796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.912046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.912296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.912535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.912769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.913005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.913135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.913266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.913468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.913701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.913974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.914223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.914473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.914720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.914967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.915200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.915450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.915691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.915931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.916168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.916406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.916655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.916907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.917147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.917404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.917640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.917909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.918153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.918394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.918780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.919036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.919286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.919525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.919769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.920016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.920277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.920544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.920767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.921021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.921277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.921515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.921766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.922033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.922277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.922517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] role.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.922768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] role.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.923029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.923880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.924136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.924392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.924643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.924894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.925145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.925394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.925644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.925902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.926157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.926408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.926657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.926905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.927344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.927586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.927840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.928093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.928347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.928601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.928841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.929099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.929357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.929601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.929853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.930094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.930334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.930589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.930848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.931109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.931349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.931587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.931824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.932061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.932301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.932550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.932788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.933045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.933295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.933536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.933782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.934062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.934314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.934556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.934796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.935033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.935282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.935531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.935765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.936009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.936308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.936571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.connection = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.936818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.937068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.937308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.937552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.937790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.938057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.938296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.938535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.939316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.939543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.939791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.940027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.940277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.940519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.940766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.941001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.941243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.941480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 12:36:07.941652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91820) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 12:36:07.943299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.943534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.966094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.966328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.966556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.975649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.995657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.995867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:07.996419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.032119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.046147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.046384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.068826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.069063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.110079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.110313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.111875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.112060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.112282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.112499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.112718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.136946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.137629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.148335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.148608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.161900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91819) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.162760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91820) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 12:36:08.180114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.180362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 12:36:08.180448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.180624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 12:36:08.180732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.180934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 12:36:08.181017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 12:36:08.181237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.181968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.182062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.182222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.192262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.192519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.192603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.192792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.192887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.193059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.193140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.193340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.200017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.200542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.204713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.204992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.205272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.205352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.205552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.205636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.205853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.205937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.206914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.207982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.230603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.231014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.232074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.232357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.232502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.232774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.237466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.237744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.237920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.238044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.238203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.238487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.244625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.244910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.245130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.245415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.249349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.249588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.249672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.249907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.250828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.251010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.251106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.251349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.251441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.251629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.267849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.268026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.268137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.268309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.272229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.272615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.275503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.275840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.278268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.278766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.280923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.284934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.286447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.288543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.288848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.289138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.289426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.289630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.289712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.290015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.292902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.293204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.293509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.293776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.294095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.294384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.306673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.306959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.307226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.307489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.311161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.311445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.311710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.311976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.314971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.316564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.316817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.319422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.319637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.319920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.320219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.320502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.320794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.321009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.321281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.324127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.324408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.324710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.324996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.325279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.332373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.332656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.332937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.333219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.333500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.333775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.334073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.336839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.337115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.337393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.337672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.337978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.338252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.338523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.349867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.352838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.354348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.355813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.357289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.358375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.359872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.360159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.360242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.360967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.363320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.364823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.365113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.365417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.367567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.369177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.371944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.372128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.372424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.372721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.372999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.373536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.377315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.377797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.378311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.378798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.381911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.382207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.382494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.387317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.387602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.387882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.389291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91820) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.389573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.389879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.390178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.390460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.390733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.391007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.391273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.391650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.391955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.392242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.392516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.392781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.393041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91820) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.394802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91819) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 12:36:08.395079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.395356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.395631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.395914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.396200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.396479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.396753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.397120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.397400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.397681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.397968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.398297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.398627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91819) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 12:36:08.427663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e8051ec9c0 pid: 91820 (default app) Jul 29 12:36:08.431033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8f5b4-1382-4e6f-beb3-4c40b9900573 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:08.431328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8f5b4-1382-4e6f-beb3-4c40b9900573 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:08.431607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8f5b4-1382-4e6f-beb3-4c40b9900573 None None] PATH_INFO: `/v3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:08.432392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e8051ec9c0 pid: 91819 (default app) Jul 29 12:36:08.435925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Fri Jul 29 12:36:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:16.066734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:16.067302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:16.067302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:16.071751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:16.076896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:16.077309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:16.077468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:16.148377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: INFO dbcounter [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] Registered counter for database keystone Jul 29 12:36:16.169207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG oslo_db.sqlalchemy.engines [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91820) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 12:36:16.175400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writer thread running {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 12:36:16.351905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG passlib.handlers.bcrypt [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91820) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 12:36:16.352234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG passlib.handlers.bcrypt [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91820) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 12:36:16.406594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:16.408239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:16.531041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 12:36:16.640251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:16.710317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: INFO dbcounter [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] Registered counter for database keystone Jul 29 12:36:16.730157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG oslo_db.sqlalchemy.engines [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91819) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 12:36:16.737122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writer thread running {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 12:36:16.860461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c6c70-e842-4db0-b14e-aefa991b1610 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:16.864924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:16.865593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:16.865908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:16.866195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:16.874422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:16.927288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 12:36:16.927604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccee16f-04dc-4648-8794-37ea3a682ea6 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:16.937623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Fri Jul 29 12:36:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:36:16.944128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:16.959053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f4709-7a89-4438-b580-95d7af2c2a41 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:16.963577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:16.964667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:16.964955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:16.965229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] PATH_INFO: `/v3/projects/9e5a5cdc1deb42199be2e349eba98664` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:16.966766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] RBAC: Authorizing `identity:get_project(project_id=9e5a5cdc1deb42199be2e349eba98664)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:17.016066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 12:36:17.016971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758da1b-1ef4-432b-970b-db0badd0e50f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:17.021566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Fri Jul 29 12:36:16 2022] GET /identity/v3/projects/9e5a5cdc1deb42199be2e349eba98664 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:18.985948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bce364-2548-4c7d-8e8a-aed3d1d507ad None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:18.986484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bce364-2548-4c7d-8e8a-aed3d1d507ad None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:18.986484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bce364-2548-4c7d-8e8a-aed3d1d507ad None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:18.991325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:18.996545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:18.996902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:18.997101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:19.016910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:19.018463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:19.026351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 12:36:19.048862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bce364-2548-4c7d-8e8a-aed3d1d507ad None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:19.060833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bce364-2548-4c7d-8e8a-aed3d1d507ad None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:19.065612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:19.066259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:19.066549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:19.066830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:19.074767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:19.076649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab0b3-30dc-450a-bb70-6845e531c1a7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:19.086178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Fri Jul 29 12:36:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:36:19.091292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:19.100634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2a8f4-9aa5-475d-9046-dab50cdab5fa None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:19.105006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:19.105628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:19.105951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:19.106234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:19.107594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:19.109420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3200f-5eb5-44be-b425-408be3dfc943 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:19.111037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Fri Jul 29 12:36:19 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:21.159979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c732e6-91f5-48ca-9c61-9d486a5ac6c7 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.160458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c732e6-91f5-48ca-9c61-9d486a5ac6c7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.160554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c732e6-91f5-48ca-9c61-9d486a5ac6c7 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.161895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:21.164572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230023e-39d5-4ae3-9058-8e4350c0d826 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.164965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230023e-39d5-4ae3-9058-8e4350c0d826 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.165136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230023e-39d5-4ae3-9058-8e4350c0d826 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.166526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:21.166885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.167177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.167455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.171419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.171704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.172613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.192016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:21.193200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:21.210444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG passlib.handlers.bcrypt [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91819) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 12:36:21.210865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG passlib.handlers.bcrypt [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91819) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 12:36:21.256956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:21.258539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:21.273214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 7/12] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:21.296934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.321116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.358924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 6/13] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:21.381335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.410372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39523990-e36b-4170-870d-e22d92e88b67 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.445202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:21.445885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.446175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.446454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.457519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:21.466406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:21.467481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:21.468489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 8/14] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:21.474458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.486233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27f42-6c8b-4f9c-9076-917ae18d7677 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.488552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.523499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:21.524748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.525035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.525598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.563310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.582143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77b065-ead6-4d28-8166-c3d4bbdd616c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.594952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:21.596751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:21.615868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:21.616491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.616774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.617053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.617783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:21.618122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:21.619500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:21.632965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 9/15] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:21.638299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.699208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 7/16] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 12:36:21.705595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.708873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.809321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9dca-8158-4807-9fbe-00cdd802b02f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.844354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:21.844534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb958b9-4b25-4839-a582-b7f06212fde7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.845198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.845743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.846046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.864389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:21.865875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:21.879354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:21.880004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:21.880287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:21.880562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:21.881736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:21.883267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:36:21.884157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:21.885053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:21.885886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:36:21.885886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:36:21.885886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:36:21.885886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:21.886115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 10/17] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:21.886179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 8/18] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:21.890236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.890856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:21.905015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.947321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:21.992730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3a30b-88dd-4980-a7cd-e047008e7063 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:21.999776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce508-7faf-4c5b-a13e-726fe900a949 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:22.029419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:22.030097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:22.030376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:22.030652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:22.031817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:22.033295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:22.034584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:22.034898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 11/19] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:22.035221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:22.035508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:22.035789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:22.036489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:22.036801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:22.038231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecfc2e-af8d-4d6f-9186-228be7d64c92 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:22.040687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:22.057879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 9/20] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:22.130326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:22.150961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242600e-5dbd-4dd9-aafe-f063fc9f5306 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:22.185768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:22.186495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:22.186692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:22.186968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:22.188300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.domain.id=default, user.id=1eea4fac81544f2bac9c90f3a5feb407) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:22.188607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:22.190102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c746f6-369e-48f6-b7dd-4a8f1fab4fbf None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:22.197481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 12/21] 10.222.0.66 () {56 vars in 1322 bytes} [Fri Jul 29 12:36:22 2022] GET /identity/v3/role_assignments?user.id=1eea4fac81544f2bac9c90f3a5feb407&scope.domain.id=default&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:25.009119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248f2b-ac36-4e1e-a3c5-571d284f83d9 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.009119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248f2b-ac36-4e1e-a3c5-571d284f83d9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.009119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248f2b-ac36-4e1e-a3c5-571d284f83d9 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.009119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 10/22] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:25.021532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f558e7-65e1-469f-92d2-2847d0298dd2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.022376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f558e7-65e1-469f-92d2-2847d0298dd2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.022810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f558e7-65e1-469f-92d2-2847d0298dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.042062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-66f558e7-65e1-469f-92d2-2847d0298dd2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:25.043500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-66f558e7-65e1-469f-92d2-2847d0298dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:25.162282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 13/23] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:25.198915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06248f2b-ac36-4e1e-a3c5-571d284f83d9 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:25.324186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7c42-fde3-42c3-a9d6-726d9408884a None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.324717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7c42-fde3-42c3-a9d6-726d9408884a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.325140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7c42-fde3-42c3-a9d6-726d9408884a None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.326735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 14/24] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:25.328585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eea7d-87a7-495d-af0c-705e64c8c61d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.329015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eea7d-87a7-495d-af0c-705e64c8c61d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.329454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eea7d-87a7-495d-af0c-705e64c8c61d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.330862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 15/25] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:25.333548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015e65-9cf0-4b69-b7cb-038a123f2f68 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.334040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015e65-9cf0-4b69-b7cb-038a123f2f68 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.334457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015e65-9cf0-4b69-b7cb-038a123f2f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.362458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06248f2b-ac36-4e1e-a3c5-571d284f83d9 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:25.370389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-23015e65-9cf0-4b69-b7cb-038a123f2f68 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:25.371852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-23015e65-9cf0-4b69-b7cb-038a123f2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:25.389743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:25.408247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:25.409139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.409578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.410063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.410947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:25.412542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e707d-5ba9-4bb5-b428-34979e4bcb84 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:25.456166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 16/26] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:25.457804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbd72-71d0-44d2-803c-837abe483364 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.458318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbd72-71d0-44d2-803c-837abe483364 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.458786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbd72-71d0-44d2-803c-837abe483364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.466134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:25.478476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c40bbd72-71d0-44d2-803c-837abe483364 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:25.480115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c40bbd72-71d0-44d2-803c-837abe483364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:25.568874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 17/27] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:25.579823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472211a-825c-4bab-a4d9-a000fdcab330 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.580081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472211a-825c-4bab-a4d9-a000fdcab330 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.580322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472211a-825c-4bab-a4d9-a000fdcab330 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.581720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 18/28] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:25.582499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472211a-825c-4bab-a4d9-a000fdcab330 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:25.663975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 11/29] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/roles => generated 219 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:25.665883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.666395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.666911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.696514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:25.703420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:25.759159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472211a-825c-4bab-a4d9-a000fdcab330 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:25.832577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:25.833143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:25.833426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:25.833701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:25.834481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:25.835911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:25.840224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:25.852119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 12/30] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:25.853048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:25.987308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bacf5-38c7-408c-88e8-2a220dfb951c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:25.991231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 19/31] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 29 12:36:25 2022] POST /identity/v3/roles => generated 225 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:25.992367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:25.998204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.053644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.054383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.054673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.054949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.069610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:26.076177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:26.077124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:26.078421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:26.079535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 13/32] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:26.088778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.193601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fa607-7e60-40cb-a77e-b69ed83249e2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.218694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c570-6341-4e4e-9413-29be888754af None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.233121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.234675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.235136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.235582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.246991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.248902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.258822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.259467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.259754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.260034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.260798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:26.261110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.262704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f123-028a-4d48-a1ef-9188dce7d8e7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f4a29149-b53e-4b5d-850c-371f349ffbcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:36:26.267946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:26.268773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:26.269548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:36:26.269548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:36:26.269548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:36:26.269548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:26.271921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 20/33] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:26.273495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6eae-16b8-4c76-9024-183f8cf56c1f None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.273985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6eae-16b8-4c76-9024-183f8cf56c1f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.274436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd6eae-16b8-4c76-9024-183f8cf56c1f None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.275893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 21/34] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:26.280242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 14/35] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:26.282581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd6eae-16b8-4c76-9024-183f8cf56c1f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.282708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.282708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.282907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.303369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:26.304675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:26.332343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.345769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.387824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 15/36] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:26.388774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.411254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd6eae-16b8-4c76-9024-183f8cf56c1f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.445480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.446141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.446433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.446717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.447419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:26.447731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.449177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bbbf8-924a-4ce4-91ca-30b0d026fea8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.465850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 22/37] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:26.467290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569a44-8d4d-43b8-aa2f-6ad840add263 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.467576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569a44-8d4d-43b8-aa2f-6ad840add263 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.467852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569a44-8d4d-43b8-aa2f-6ad840add263 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.469595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 23/38] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:26.470687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0569a44-8d4d-43b8-aa2f-6ad840add263 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.492870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec81e2-bacc-4f30-a874-715e9504a0c4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.529438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.530100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.530389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.530679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.532783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.534287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a4d9-66a3-4d37-b511-594b9d9e45af None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.535873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 16/39] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:26 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:26.537046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.537344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.537625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.554687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:26.555909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:26.576666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.579154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0569a44-8d4d-43b8-aa2f-6ad840add263 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.586426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.614268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.614840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.615116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.615392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.616535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.618017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.619594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 24/40] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:26.620326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.629300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 17/41] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:26.630757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.721106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e58f-88fe-4038-b3a5-bae8c4162448 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.732356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36a79-8e29-4ecb-8372-b75d40531c89 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.754023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.754645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.754931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.755264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.755946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.757464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.765607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.766290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.766580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.766864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.768013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.769440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.771054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 18/42] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:26.771795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.791277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 25/43] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:36:26 2022] POST /identity/v3/projects => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:26.792366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.814266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.827308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:26.907404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef526c-6c90-4bde-8d73-c7b00ec2d4ab None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.911771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe6fd4-f0ea-4537-aa9f-ade830639101 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:26.947462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.948682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.948682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.948682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.949340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.950828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093806f8-3698-4c6c-9e6f-623da4634ebe None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.951158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:26.951848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:26.952082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:26.952362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:26.953992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:26.955737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:26.973030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 26/44] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 29 12:36:26 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:26.974295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:26.990335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 19/45] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:36:26 2022] POST /identity/v3/projects => generated 312 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:27.093053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:27.118721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04d89d-85f4-442c-8d6f-9acdad67a43f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:27.158684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:27.159295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:27.159578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:27.159881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] PATH_INFO: `/v3/domains/default/users/1eea4fac81544f2bac9c90f3a5feb407/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:27.161951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1eea4fac81544f2bac9c90f3a5feb407, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:27.163406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa09ec-12bf-4acd-b253-1e0343d89842 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:27.179275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 27/46] 10.222.0.66 () {58 vars in 1361 bytes} [Fri Jul 29 12:36:26 2022] PUT /identity/v3/domains/default/users/1eea4fac81544f2bac9c90f3a5feb407/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:29.060243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bf298-ff0f-4c1b-bee8-acb6ee5501d2 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.060778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bf298-ff0f-4c1b-bee8-acb6ee5501d2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.060778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bf298-ff0f-4c1b-bee8-acb6ee5501d2 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.062412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 20/47] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:29.067434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367031c-a973-4d89-86d3-2b055edd80f7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.067823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367031c-a973-4d89-86d3-2b055edd80f7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.068003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367031c-a973-4d89-86d3-2b055edd80f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.086826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4367031c-a973-4d89-86d3-2b055edd80f7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:29.088329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4367031c-a973-4d89-86d3-2b055edd80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:29.156324 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 28/48] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:29.179284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bf298-ff0f-4c1b-bee8-acb6ee5501d2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.190658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc976e-b233-4295-b52a-ab23f8eceefe None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.191018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc976e-b233-4295-b52a-ab23f8eceefe None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.191343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc976e-b233-4295-b52a-ab23f8eceefe None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.192883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 29/49] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:29.197611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.197936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.198223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.216211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:29.217397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:29.261931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:29.288701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bf298-ff0f-4c1b-bee8-acb6ee5501d2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.290128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 30/50] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:29.312390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.320099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:29.325551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.326228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.326518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.326795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.328456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:29.329940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:29.331530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 21/51] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:29.337083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.415138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114a376-c694-44f8-8ef8-72ee9af1ba63 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.436430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e01d-2c59-45f7-81b5-9e7c0f01a422 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.449891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.450539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.450822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.451101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.461571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:29.467092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:29.467935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:29.468694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 31/52] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:29.470716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.471338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.471628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.471903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.472583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:29.474079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:29.474588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.489779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:29.502349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 22/53] 10.222.0.66 () {60 vars in 1140 bytes} [Fri Jul 29 12:36:29 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:36:29.544366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:29.583967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc46a-448f-4f67-a14e-c06aa82c2fc5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.620074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.620520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.620806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.621085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.621861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:29.622179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:29.623597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:29.632885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 32/54] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:29.638554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.741080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99912e49-5645-4e29-a7ac-b0ab0e4399f5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.778183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.778837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.779125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.779407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.780583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:29.782599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:29.784369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 23/55] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:29.789613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.888376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c0e43-e7f8-4d1d-a2dd-7f6059fb68f1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:29.914733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:29.923111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:29.923762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:29.924048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:29.924328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:29.925475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:29.926980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd51a2f-aa2c-4dae-8c41-104f42b25203 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:29.928610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 33/56] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:29.934457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:29.996706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:30.033802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b637c-d407-427b-b742-1d711cf16fda None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:30.067260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:30.067862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:30.068149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:30.068426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:30.069219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.domain.id=default, user.id=1eea4fac81544f2bac9c90f3a5feb407) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:30.069532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:30.070999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c38525-64a8-4bfe-9f86-a010f4b1b30d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:30.079518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 24/57] 10.222.0.66 () {56 vars in 1322 bytes} [Fri Jul 29 12:36:29 2022] GET /identity/v3/role_assignments?user.id=1eea4fac81544f2bac9c90f3a5feb407&scope.domain.id=default&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:31.520402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399707-d879-4362-915b-61ba6f7f422d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:31.520909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399707-d879-4362-915b-61ba6f7f422d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:31.520909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399707-d879-4362-915b-61ba6f7f422d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:31.522288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 34/58] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:31.527252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:31.527620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:31.527811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:31.546539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:31.547711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:31.612990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 25/59] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:31.638365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce399707-d879-4362-915b-61ba6f7f422d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:31.738717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce399707-d879-4362-915b-61ba6f7f422d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:31.770923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:31.771499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:31.771786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:31.772065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:31.773210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:31.774706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e3ae3-472d-4e47-b63b-38374a7d006b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:31.776263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 35/60] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:31.782256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:31.837349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:31.878290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4afc4-be0d-4830-98ab-12f4d5049e3f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:31.910862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:31.911477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:31.911762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:31.912040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:31.912728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:31.914202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4e1a2-4fbd-4e87-a59b-bf48e09a83f9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:31.940225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 26/61] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:31 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:35.179726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb57d1-83c9-412a-bf44-4017e357a632 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.180814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb57d1-83c9-412a-bf44-4017e357a632 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.181294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb57d1-83c9-412a-bf44-4017e357a632 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.182898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 36/62] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:35.188649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b59f-6538-4c81-a233-d65c5a434593 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.189526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b59f-6538-4c81-a233-d65c5a434593 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.190035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b59f-6538-4c81-a233-d65c5a434593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.218424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f401b59f-6538-4c81-a233-d65c5a434593 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:35.219945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f401b59f-6538-4c81-a233-d65c5a434593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:35.223408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ea5c-4474-4c4e-901a-6ffa98fe13e6 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.223948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ea5c-4474-4c4e-901a-6ffa98fe13e6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.224423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ea5c-4474-4c4e-901a-6ffa98fe13e6 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.225986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 37/63] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:35.230969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.234074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.234074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.251785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:35.253102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:35.314860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 27/64] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:35.316704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2bb64-da3e-427f-807c-4b1c0a4dcfe4 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.317012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2bb64-da3e-427f-807c-4b1c0a4dcfe4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.317744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2bb64-da3e-427f-807c-4b1c0a4dcfe4 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.319433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 28/65] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:35.319626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.325835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.326131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.326414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.337071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 38/66] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:35.349244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:35.350765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:35.361103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.402335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.438520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 29/67] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:35.440155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.487767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cde9d-e6b5-4fb4-81cb-b6faf81a9845 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:35.535207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:35.535890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.536177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.536453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.547806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:35.552801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:35.553648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:35.554437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 39/68] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:35.555052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.570099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a270206-be07-4089-93ec-661eb83339d4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:35.606740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:35.607427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.607718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.607997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.616744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.620041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-f195eaa4-e625-401e-a078-ab7477f612c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:35.625600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:35.626473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:35.627246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 30/69] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:35.628485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d855-e81d-4741-947e-5add5e7c69d5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.628787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d855-e81d-4741-947e-5add5e7c69d5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.629071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d855-e81d-4741-947e-5add5e7c69d5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.630542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 31/70] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:35.631162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d855-e81d-4741-947e-5add5e7c69d5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.673902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f9bc5-6040-46dd-afbb-36a6ecb12407 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:35.675683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.710902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:35.711541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.711823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.712091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.722642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:35.727456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:35.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:35.729140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 40/71] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:35.729760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.758961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d855-e81d-4741-947e-5add5e7c69d5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:35.793369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:35.794016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.794301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.794578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.795292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:35.795600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:35.797540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da38c0-c0b5-4f88-88c0-93bdd8088fcb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:35.805349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 32/72] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:35.806509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518180d-b38c-4531-a205-0c0c14451bd3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.806795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518180d-b38c-4531-a205-0c0c14451bd3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.807089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518180d-b38c-4531-a205-0c0c14451bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.825051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c518180d-b38c-4531-a205-0c0c14451bd3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:35.826256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c518180d-b38c-4531-a205-0c0c14451bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:35.831814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecedf95-5576-4912-9320-f358d2583714 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:35.846393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.867228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:35.867848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.868135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.868417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.869133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:35.869443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:35.870851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:35.879010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 41/73] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:35.879010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.893508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 33/74] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:35.894858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634e7f-ced1-4831-9b21-f6afc322070c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:35.895156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634e7f-ced1-4831-9b21-f6afc322070c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:35.895441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634e7f-ced1-4831-9b21-f6afc322070c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:35.896696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 34/75] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:35.897387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da634e7f-ced1-4831-9b21-f6afc322070c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:35.929646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:35.980943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30202283-521e-4927-96a3-dd42e74dd31c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.003113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da634e7f-ced1-4831-9b21-f6afc322070c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.014722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.015871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.016159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.016435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.017148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:36.017455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.018898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.026660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 42/76] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.031391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.037504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.038179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.038465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.038747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.039931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.041401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb4be-38e5-42e9-8edd-0c221df2fb12 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.043033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 35/77] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:35 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.044164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.044423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.044703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.062735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:36.064020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:36.068641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.129221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2229-a717-4f56-8df8-8cecdeb53d89 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.134799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 36/78] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:36.138823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.164782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.164943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.165836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.166216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.166498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.181682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.183133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.184780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 43/79] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.185552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.241591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd91ff1-dabf-4e86-b838-f2423b646ac1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.275489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.276142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.276449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.276728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.283137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.287477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:36.289515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458edea-c1f3-4542-8a5e-085585828086 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:36.292399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:36.293429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 37/80] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:36.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.323732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.324378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.324664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.324945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.326146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.327625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.329236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 44/81] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.330733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.383168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.401056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce3e2-fe2f-49e2-b8ec-ec07b4ca5cfb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.437359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e72a-5d7a-40b2-b382-f953b465faad None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.437522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.438072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.438361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.438638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.471143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.471785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.472076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.472395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.473667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.475315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.477107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 45/82] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.478713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.505555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.507083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.511243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.581363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2e960-8431-46f5-b21b-c2749067009a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.587394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.605027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 38/83] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:36.606075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.616710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.617339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.617625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.617938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.684001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.685487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.704304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd34b-6b09-42d3-bad8-78661dbe4d3e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.726697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.741647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.742333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.742614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.742889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.743602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:36.743907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.745290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.753491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 39/84] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:36.754332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.800368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.803955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 46/85] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:36.804835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:36.857948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338648dd-2c9d-4859-a828-16ec9187f780 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.891063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.891685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.891965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.892241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.903102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c671-b9f6-47a7-8ed2-1d492f5a417a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:36.933706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:36.939010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:36.939622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:36.939909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:36.940190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:36.940875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.942397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.959810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:36.961389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:36.970443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 47/86] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:36 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:36.971289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.041258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.097518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dd2f7-e099-4751-b59b-3b4d2080d0d5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.104555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 40/87] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:37.105435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.150237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.150905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.152715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.154226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.156088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 48/88] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:36 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.157276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.228719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26302-866a-4ad8-beeb-55c6539599f0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.231447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.275152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.275822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.276112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.276394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.277669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.279297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.281648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 41/89] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.286615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.293193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb57538-0341-4efd-84a1-13f1245a5469 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.307689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.334146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.334800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.335088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.335367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.336542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.338024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.340021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 49/90] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.340802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.415218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2951-af7d-4a4f-8264-274c8dfaf937 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.456724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fe62-5ffb-4739-8aee-0a5d9e59fd2b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.456991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.458104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.458890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.459274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.460524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.462019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.464020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 42/91] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.469409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.482145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.494453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.495334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.495721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.496155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.497057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=1eea4fac81544f2bac9c90f3a5feb407) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:37.497597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.499954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.506437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 50/92] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/role_assignments?user.id=1eea4fac81544f2bac9c90f3a5feb407&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.507337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.561052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.581702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d313f31-984c-47bd-86f5-8bd8eff9c5fa None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.626656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.627321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.627608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.627889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.695311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aafca-9ad6-4b3e-bc0a-8d98e83d2603 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:37.744740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.746898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.763341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:37.763982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:37.764270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:37.764639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:37.765472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:37.765795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:37.776322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:37.787933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 51/93] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:37.788692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.851698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:37.873229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 43/94] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:37.881527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:37.933611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:38.013046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66eb15-e0db-41c0-a3b8-c24a40c9cc2b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:38.082331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545f6d0-ca2c-43a5-9c46-9da41746b852 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:38.083580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:38.084442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:38.084946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:38.085436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:38.086532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:38.087048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:38.088723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:38.100118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 52/95] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=6dda70e53deb474a96e637c5fff03848 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:38.145194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:38.146295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:38.146814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:38.147322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] PATH_INFO: `/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:38.148967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] RBAC: Authorizing `identity:get_project(project_id=0a850f2d6c8b4c309175f5ab4d3817c0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:38.150737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b2845-2543-4703-bd4f-56fdde774595 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:38.152740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 44/96] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:37 2022] GET /identity/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:38.158778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:38.277698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b9803-bad3-4cbd-a1bc-6b8364cff663 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:38.296480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:38.317234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:38.318280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:38.318768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:38.319242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:38.320356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=0a850f2d6c8b4c309175f5ab4d3817c0, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:38.320880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:38.322594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab50b-823a-40e9-9b41-acc8d4f95f88 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:38.329697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 53/97] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:38 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=0a850f2d6c8b4c309175f5ab4d3817c0&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:39.464476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e9463-0d90-4735-8a0d-7f969b461817 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:39.465455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e9463-0d90-4735-8a0d-7f969b461817 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:39.465997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e9463-0d90-4735-8a0d-7f969b461817 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:39.467569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 45/98] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:39.473557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:39.474439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:39.474917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:39.498807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:39.500887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:39.596718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 54/99] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:39.631054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e9463-0d90-4735-8a0d-7f969b461817 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:39.700213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:39.770307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e9463-0d90-4735-8a0d-7f969b461817 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:39.811565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:39.812548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:39.813060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:39.813532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:39.836742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:39.842050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:39.843545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:39.844324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:39.852744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 46/100] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:39.854560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:39.968637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06176b0-cb26-45e6-8c43-269183636be1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:39.980217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:40.012097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.012724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.013010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.013290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.014038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:40.014350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.015784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049c4a2-0f34-4b51-ad0f-4ab158198b0e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.024286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 55/101] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:40.029732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:40.071774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b4b4-df8f-4cdb-a6ee-5cfae15c6538 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.072079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b4b4-df8f-4cdb-a6ee-5cfae15c6538 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.072340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b4b4-df8f-4cdb-a6ee-5cfae15c6538 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.073991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 56/102] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:40.091148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.091493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.091772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.120627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:40.121649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:40.221322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a7c42-d092-4b06-a5b3-4d69d430e7e0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:40.250197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:40.273959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 57/103] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:40.286275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.286912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.287253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.287538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] PATH_INFO: `/v3/users/31a367eff97b465091c62aa43c85cd64` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.314754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] RBAC: Authorizing `identity:get_user(user_id=31a367eff97b465091c62aa43c85cd64)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.317207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.319168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:40.319984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 47/104] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:40 2022] GET /identity/v3/users/31a367eff97b465091c62aa43c85cd64 => generated 289 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:40.325418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:40.425656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce2762-7957-4864-900b-27ba0b7bb47c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:40.438189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:40.461937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.462647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.462942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.463224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.464409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.465897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.467787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 58/105] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:40.473470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:40.491273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078061-1a26-4fd1-b860-78eb6905f912 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:40.538631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.539201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.539485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.539763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.576327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a90c2-fa6d-4094-be04-25f459b31d24 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:40.601081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:40.630526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.631174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.631464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.631746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.632561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.634110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.637630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.639138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.680210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 59/106] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:36:40 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:36:40.749444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 48/107] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:40.755043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:40.810320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:40.916380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85829945-9e95-4c6d-b3c3-471b16ef6b40 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:40.954866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:40.955909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:40.956395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:40.956859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:40.958260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:40.960126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61711cee-2771-4b1c-a0a8-ba45292e9810 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:40.962258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 60/108] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:40 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:40.968300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.072563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.095238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b6413-cada-4c7e-be55-d409f903706d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.095579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b6413-cada-4c7e-be55-d409f903706d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.095799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b6413-cada-4c7e-be55-d409f903706d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.097299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 61/109] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:41.103938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.104228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.104514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.116160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32393ade-f385-4104-84b2-75786de9f13f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.143601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:41.144803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:41.153613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.154275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.154560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.154838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.155684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=31a367eff97b465091c62aa43c85cd64) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:41.155998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:41.157463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491d515-563b-43ea-9cca-7b5116ec90e5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:41.163640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 49/110] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:40 2022] GET /identity/v3/role_assignments?user.id=31a367eff97b465091c62aa43c85cd64&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:41.164920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c684-d7fc-4def-87bf-3562f0fd21fa None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.165301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c684-d7fc-4def-87bf-3562f0fd21fa None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.165592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c684-d7fc-4def-87bf-3562f0fd21fa None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.178223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 50/111] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:41 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:41.183950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c905-ef0b-485d-bfb4-de2aa2c20fce None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.183950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c905-ef0b-485d-bfb4-de2aa2c20fce None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.183950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c905-ef0b-485d-bfb4-de2aa2c20fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.218970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3d93c905-ef0b-485d-bfb4-de2aa2c20fce None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:41.220453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3d93c905-ef0b-485d-bfb4-de2aa2c20fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:41.232168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 62/112] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:41.256911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.282257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.302769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 51/113] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:41.305439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8116-e4e6-4232-86af-01a41c3c4358 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.305735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8116-e4e6-4232-86af-01a41c3c4358 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.306045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8116-e4e6-4232-86af-01a41c3c4358 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.307558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 52/114] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:41.314045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.314565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.314855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.339738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:41.341708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:41.411373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.420338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a50d5-cbe0-4116-94b6-890adfbeecff None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.425625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 53/115] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:41.426597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.503110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.503782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.504063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.504334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.520172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:41.526118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:41.527074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:41.527868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:41.530992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 63/116] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:41.532151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.546726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd35387-a247-4f24-bb27-bd9c516e1a59 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.588050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.589075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.589572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.590137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.601970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d1719f36-ed63-4081-a95b-242e762c0ea5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:41.607334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:41.608226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:41.609032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:41.611411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 54/117] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:41.617555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50c25-0d8b-47c4-bb46-827ffe1cf90f None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.618197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50c25-0d8b-47c4-bb46-827ffe1cf90f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.618713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50c25-0d8b-47c4-bb46-827ffe1cf90f None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.620794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 55/118] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:41.621708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a50c25-0d8b-47c4-bb46-827ffe1cf90f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.641758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.666835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcc01e-9cd9-4a69-bd68-ee18581a68c7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.704440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.708822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.710277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.710641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.711043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.723933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:41.729580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:41.730472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:41.731284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 64/119] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:41.731958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.768186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a50c25-0d8b-47c4-bb46-827ffe1cf90f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.806555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.807135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.807422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.807706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.808545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:41.808854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:41.810289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c05e2-f411-4ec8-96b1-921ecf925431 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:41.819641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 56/120] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:41.820999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.821250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.821581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.841673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:41.843025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:41.876357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bf664-83ac-4cd2-8ea3-4ba3152bf419 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:41.922052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 57/121] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:41.923222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.928280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:41.928989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:41.929350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:41.929698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:41.930550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:41.930933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:41.932474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:41.951220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 65/122] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:41.956907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:41.959847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:41.990864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.055886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d31a11-3cbf-444d-b673-256a92432332 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.096777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.097328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39998-9457-4e66-a33f-a08171b3fda0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.098004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.098513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.098995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.099917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:42.100421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.102031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.111429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 58/123] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.112470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.162030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.162722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.163413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.163760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.165269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.167141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.169559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 66/124] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:41 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.171291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.232514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62422ecb-668c-4867-bf9f-95b2f2ac8bd7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.240441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.271535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.272519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.273027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.273514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.274964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.275200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.277437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.279369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 59/125] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.280475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.325271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a35a1-0c3d-4282-9ab5-9008949198e7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.377102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.378009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.378520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.378966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.392020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:42.398079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b2681-93af-488e-9bc4-922ac5b33b19 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:42.398398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:42.399282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:42.400048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:42.402728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 67/126] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:42.403733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.442133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.442794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.443081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.443358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.444530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.446026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.448201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 60/127] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.452957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.498577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.550819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e64ccd-b8e9-4b70-953a-8633c54c6f5a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.567544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4112-76ff-4e13-8207-a22971d2b057 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.604138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.605359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.605901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.606517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.606977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.607759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.608054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.608492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.608946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.609881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.611042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.611440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.613121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 68/128] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.613586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 61/129] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.621958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.622101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.622176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.748029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35b51-2ffb-461b-a229-a9da4425abab None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.749524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dde5cd-06ab-498c-833a-af30bb8d4fc2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.760365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.786612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.787525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.787829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.788179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.788466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.788666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.788770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.789333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.789967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.790479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:42.790964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.791431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.792711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.798497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 62/130] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.799346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.803565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 69/131] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:42.804668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.869921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:42.914594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803345-7b79-4da9-ac75-94dfc5db8f72 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.915279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e438da-91ea-447b-9336-06c198f4787a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:42.950848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.951532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.951893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.952211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/4c88a9c9720f4b08add7dd0deda4313d/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.952642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:42.953781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:42.954387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:42.954845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/4c88a9c9720f4b08add7dd0deda4313d/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:42.955474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user_id=4c88a9c9720f4b08add7dd0deda4313d, role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.957247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user_id=4c88a9c9720f4b08add7dd0deda4313d, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:42.958258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.959170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:42.981999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 63/133] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:42 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/4c88a9c9720f4b08add7dd0deda4313d/roles/6dda70e53deb474a96e637c5fff03848 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:42.981999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:42.982628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 70/133] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:42 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/4c88a9c9720f4b08add7dd0deda4313d/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:42.984353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.038229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:43.115040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cd18f-232b-409a-9a1b-174a92547503 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.156961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.157682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.158005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.158290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/1eea4fac81544f2bac9c90f3a5feb407/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.160423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user_id=1eea4fac81544f2bac9c90f3a5feb407, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.162071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.180635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 64/134] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:42 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/1eea4fac81544f2bac9c90f3a5feb407/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:43.192050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173bb63-1b4d-4f0f-bea0-98d20c7e1325 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.229155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:43.253068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.253762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.254082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.254368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.255982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.257511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9a1b7-1410-4c95-a5af-8ac2cb75cc8c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.259739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 71/135] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:42 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:43.265492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.392127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:43.393841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dcbdc-5ab1-40bd-895e-7b82f6b89ed9 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.394255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dcbdc-5ab1-40bd-895e-7b82f6b89ed9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.395017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dcbdc-5ab1-40bd-895e-7b82f6b89ed9 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.396045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 72/136] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:43.398354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810fc6-773d-4d84-91fd-37eac0c8066e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.403430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8716-c3bb-4e97-b5f6-454737f121da None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.403781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8716-c3bb-4e97-b5f6-454737f121da None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.404064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8716-c3bb-4e97-b5f6-454737f121da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.443726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e12a8716-c3bb-4e97-b5f6-454737f121da None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:43.444946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e12a8716-c3bb-4e97-b5f6-454737f121da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:43.452094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.453134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.453666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.454219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] PATH_INFO: `/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.455813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] RBAC: Authorizing `identity:get_project(project_id=0a850f2d6c8b4c309175f5ab4d3817c0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.458180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.460160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 65/137] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:43 2022] GET /identity/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:43.467565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.531741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 73/138] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:43.539093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f343d-22e0-4c08-b89a-5c7b50842222 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.539385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f343d-22e0-4c08-b89a-5c7b50842222 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.539681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f343d-22e0-4c08-b89a-5c7b50842222 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.541141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 74/139] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:43.559931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f343d-22e0-4c08-b89a-5c7b50842222 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.607774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29560-e8c4-4c43-a164-21585d3beff1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.652892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.653838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.654146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.654435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] PATH_INFO: `/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0/users/4c88a9c9720f4b08add7dd0deda4313d/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.657766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a850f2d6c8b4c309175f5ab4d3817c0, user_id=4c88a9c9720f4b08add7dd0deda4313d, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.659439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333d196-7a58-4da8-b6b2-ae550034e577 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.678068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 66/140] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:43 2022] PUT /identity/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0/users/4c88a9c9720f4b08add7dd0deda4313d/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:43.679567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.680209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.680589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.693054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f343d-22e0-4c08-b89a-5c7b50842222 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.707463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:43.709151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:43.716553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:43.750304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:43.751419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.752235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.752523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.754332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.755740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.757402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.762591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 75/141] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:43.773230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.856557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 67/142] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:43.924736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c817a-ab4f-43ca-b46d-eab801a2b9fc None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:43.955350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:43.961443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:43.962143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:43.962439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:43.962723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:43.963447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:43.964936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:43.996315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 76/143] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:43 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:36:44.136042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814c39-575f-4c36-a75a-094a583ed01f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:44.176266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:44.177361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:44.177901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:44.178419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:44.191506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:44.197380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:44.198439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:44.199426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:44.201447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 68/144] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:44.208269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:44.266401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:44.329333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188293-17db-4bd0-a1f2-01f8800edda2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:44.365552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:44.366258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:44.366826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:44.367104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:44.367908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:44.368223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:44.369708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:44.378841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 77/145] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:44.384833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:44.406925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:44.496008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ef5e-9094-4a06-8c66-0d9904d8d029 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:44.533253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:44.533933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:44.534249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:44.534569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] PATH_INFO: `/v3/users/31a367eff97b465091c62aa43c85cd64` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:44.535853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] RBAC: Authorizing `identity:get_user(user_id=31a367eff97b465091c62aa43c85cd64)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:44.537353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:44.539055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 69/146] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:44 2022] GET /identity/v3/users/31a367eff97b465091c62aa43c85cd64 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:44.544853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:44.655768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6540147-5561-4390-b566-eaf1ee79240f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:44.658244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:44.693799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:44.694507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:44.694788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:44.695067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:44.696336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:44.697923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c279fa8-3c82-465f-a45c-ad88b0d2b0be None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:44.699885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 78/147] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:44 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:44.706200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:44.798915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:44.817416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5a54-d123-4207-82b5-137d45ed07fe None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:44.857003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:44.857700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:44.858541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:44.858827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/31a367eff97b465091c62aa43c85cd64/roles/a4543dd62d2744a08bf092cf6c7cce33` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:44.860845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user_id=31a367eff97b465091c62aa43c85cd64, role_id=a4543dd62d2744a08bf092cf6c7cce33)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:44.862410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbbeeb-f190-42fe-9494-8da06b926736 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:44.876890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 70/148] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:44 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/users/31a367eff97b465091c62aa43c85cd64/roles/a4543dd62d2744a08bf092cf6c7cce33 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:46.039113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f854d0-341b-4d1b-bacf-76a0f6819486 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.039623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f854d0-341b-4d1b-bacf-76a0f6819486 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.039889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f854d0-341b-4d1b-bacf-76a0f6819486 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.041381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 79/149] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:46.055067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.055527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.055708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.078719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:46.080004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:46.185789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 71/150] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:46.227891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f854d0-341b-4d1b-bacf-76a0f6819486 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:46.347231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f854d0-341b-4d1b-bacf-76a0f6819486 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:46.383619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:46.384338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.384639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.384937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.396417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:46.401500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:46.402440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:46.403266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:46.405601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 80/151] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:46.411651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:46.491726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:46.520816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544883e-f3e0-4cfc-be07-ca520708b844 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:46.558279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:46.558917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.559268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.559556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.560289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:46.560602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:46.562112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:46.570832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 72/152] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:46.576510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:46.613051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:46.688229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a07528-7bda-4382-a81b-21dc6bad32f1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:46.725745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:46.726503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.726790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.727119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.728389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:46.729930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ad29d-54ff-42d3-8a5a-f3891a1498fc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:46.731635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 81/153] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:46 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:46.737406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:46.854228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af5ba6-aa0f-417d-a769-d9f9c6872a3d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:46.877147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265b34c-c51d-4fee-913c-56ff2a1d8429 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.877424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265b34c-c51d-4fee-913c-56ff2a1d8429 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.877614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265b34c-c51d-4fee-913c-56ff2a1d8429 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.879151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 82/154] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:46.888896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.889339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.889626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.911515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:46.912284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:46.912913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:46.913016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:46.913192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:46.913465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:46.923160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:46.992370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 83/155] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 29 12:36:47.021385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:47.098181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:47.138772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:47.140725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cfec9-c0c9-44f3-9cf0-db0d208ddb34 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:47.274805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0624042-a936-4da6-9589-82f5c1dc3fbc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:47.367438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:47.368088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.368369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.368647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.392950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 73/156] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:47.398041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17afd9-382c-45a0-8397-15425c4ad80c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.398344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17afd9-382c-45a0-8397-15425c4ad80c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.398636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17afd9-382c-45a0-8397-15425c4ad80c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.400093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 74/157] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:47.401080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dd7c-9caa-4d7c-b31c-641aa5145771 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.401372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dd7c-9caa-4d7c-b31c-641aa5145771 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.401650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dd7c-9caa-4d7c-b31c-641aa5145771 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.402405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:47.402942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 75/158] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:47.406134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8dd7c-9caa-4d7c-b31c-641aa5145771 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-05241c8f-438e-4e41-900d-b6c2e0e82977 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:47.410461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:47.411391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:47.412135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 84/159] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:47.412798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea1ac-23c4-4152-b896-ba24d311240e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.413061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea1ac-23c4-4152-b896-ba24d311240e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.413481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbea1ac-23c4-4152-b896-ba24d311240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.449527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ffbea1ac-23c4-4152-b896-ba24d311240e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:47.450970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ffbea1ac-23c4-4152-b896-ba24d311240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:47.522193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:47.630298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 85/160] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:47.631469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.633267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.633556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.672547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:47.673648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:47.686770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8dd7c-9caa-4d7c-b31c-641aa5145771 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:47.709638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:47.769709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:47.770769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:47.771260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:47.771730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:47.774670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:47.776476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:47.778465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 76/161] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:47 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:47.779470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:47.844022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 86/162] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:47.846037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:47.977058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5236b-7dfc-44e0-aa62-c102e5a71cad None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:47.983454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.037840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e79f8-c22d-46af-8303-d9cbfb5f7d72 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.043464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.044327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.044937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.045959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.051392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:48.051965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.053551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.065369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 77/163] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:48.068032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.095985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.097005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.097485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.097987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.119577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:48.126147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:48.126972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:48.127745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:48.135384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 87/164] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:48.136551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.162734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.294964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65566721-c88d-45d0-95fd-2b8f9fe7d4b8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.316940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70783e-340d-4e65-9fd5-f8a245326394 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.357436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.358186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.358472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.358746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.359544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:48.359853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.361295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.375676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 78/165] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=6dda70e53deb474a96e637c5fff03848 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:48.376512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.391522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.392445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.392905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.393330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.411281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:48.417414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:48.418598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:48.419481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:48.422499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 88/166] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:48.423680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.457128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.505929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.536553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894b7e6-6864-4dca-b128-cb61d76a5bff None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.541968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c112e4-acc3-42bf-9690-af7ead2c1ab9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.577481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.577800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.578224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.578608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.580160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.581442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ed575-cd5d-405c-99db-0843cc564d85 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.583688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 79/167] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:48.584971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed022ba4-eaa8-4b20-aae0-b84f14a7349b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.585337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed022ba4-eaa8-4b20-aae0-b84f14a7349b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.585685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed022ba4-eaa8-4b20-aae0-b84f14a7349b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.587064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 80/168] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:48.590167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.590956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.591244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8491-1ff5-4a09-95cf-93af4f392227 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.591528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.591658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8491-1ff5-4a09-95cf-93af4f392227 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.592059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8491-1ff5-4a09-95cf-93af4f392227 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.592782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.593642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:48.594108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.595931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.597014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 81/169] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:48.599335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d8491-1ff5-4a09-95cf-93af4f392227 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.608907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 89/170] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:48.610024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:48.720259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.726444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d8491-1ff5-4a09-95cf-93af4f392227 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.756746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25459-0793-4991-9247-5c7f4bc4e6f4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:48.772099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.772735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.773020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.773297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.774056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:48.774367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.776543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a107-c884-4f36-9dd3-c0b89fa635b6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.787979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 82/171] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:48.789980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97da6d-8448-45ca-b5cd-3205a2ed982a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.790272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97da6d-8448-45ca-b5cd-3205a2ed982a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.790562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97da6d-8448-45ca-b5cd-3205a2ed982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.807721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:48.813482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:48.814461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.814965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.815443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.817139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fd97da6d-8448-45ca-b5cd-3205a2ed982a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:48.818378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fd97da6d-8448-45ca-b5cd-3205a2ed982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:48.890228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:48.892066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:48.901800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 83/172] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:48.907165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:48.907675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:48.908157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:48.932694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:48.933890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:49.007989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 84/173] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:49.008881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.024038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 90/174] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:49.029234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.039611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.060676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.122115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa9e8b-84b0-42ca-ac23-037064aef8f7 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.133033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fa55c-2939-4055-bb98-d7a8872febd1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.156416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.156992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.157272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.157545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] PATH_INFO: `/v3/users/1eea4fac81544f2bac9c90f3a5feb407` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.158721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] RBAC: Authorizing `identity:get_user(user_id=1eea4fac81544f2bac9c90f3a5feb407)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.160155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2a1a-ed43-4bb5-8ce4-8f4e29b1e532 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.161737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 85/175] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/users/1eea4fac81544f2bac9c90f3a5feb407 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:49.162861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7352-6d97-4702-92b4-56ec38887db1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.163140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7352-6d97-4702-92b4-56ec38887db1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.163414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7352-6d97-4702-92b4-56ec38887db1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.164617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 86/176] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:49.165296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb7352-6d97-4702-92b4-56ec38887db1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.169946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.170666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.170956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.171226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] PATH_INFO: `/v3/users/4c88a9c9720f4b08add7dd0deda4313d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.172438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] RBAC: Authorizing `identity:get_user(user_id=4c88a9c9720f4b08add7dd0deda4313d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.173898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.175493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 91/177] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/users/4c88a9c9720f4b08add7dd0deda4313d => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:49.176242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.259236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.272837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb7352-6d97-4702-92b4-56ec38887db1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.280152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.283281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43ea9e-0f4f-4ee0-86b5-1cd72944923c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.307225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.307869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.308151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.308428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.318117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.318762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.319046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.319133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:49.319322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.320483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.321959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.323625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 92/178] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e9594313-5cfd-4b84-8919-9908ed655c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:49.323989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:49.325183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:49.331314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:49.331917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.332008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 87/179] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:49.332008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15f42d-c6f2-4ee5-854e-b3033d566643 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.332008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15f42d-c6f2-4ee5-854e-b3033d566643 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.332008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15f42d-c6f2-4ee5-854e-b3033d566643 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.332008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 88/180] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:49.333128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15f42d-c6f2-4ee5-854e-b3033d566643 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.428606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abbb67-e4f1-4aef-a9e5-d98906007ddf None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.432745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15f42d-c6f2-4ee5-854e-b3033d566643 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.461540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.462225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.462509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.462780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.465926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.466537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.466820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.467189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.473865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-114ba19c-6577-4cf8-a4c2-f436ff1813e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:49.479122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:49.479981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 93/181] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:49.481616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.482348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.482348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.488384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.499965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:49.501249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:49.524755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.540320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.541761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.568924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 94/182] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:49.569751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.646823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 89/183] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:49.647658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.671152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef7cc2-2da1-4f65-a046-df55c53a69d5 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.695132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.705560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.706216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.706494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.706768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.733681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.747219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8b6-c09b-4740-bb78-10d716f4392d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.785519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.786919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.787114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a629b-e6bb-4040-9f81-3326eaa09768 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.787588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.787872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.788354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.789716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:49.790148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.791612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.797739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 90/184] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:49.798571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.905521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 95/185] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:49.905521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.905521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.905521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.922621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:49.923860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:49.930595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8f94-b3dc-4c9f-ba11-1293265f4e1d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:49.946204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:49.971409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:49.973086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:49.973086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:49.973086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:49.973331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:49.974216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:49.975948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:49.984685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 91/186] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:49.985605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:49.998085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.015205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 96/187] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:50.022113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.103461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810a08-7df3-42bb-945b-e058b77fe460 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.122955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941d45c-1c4d-4d08-b7b4-fc77e883af8a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.146248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.146865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.147157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.147432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.148157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:50.148464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.149858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.158443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.158997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 92/188] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.159412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.160165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.160449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.160722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.171696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:50.176955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:50.177851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:50.178658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 97/189] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:50.179285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.198576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.229071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.268862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecd5a-6fc2-4e84-9adf-5e5a10ca0c82 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.285190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a013547-582f-486d-baf5-7187a8e96c09 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.304474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.305117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.305401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.305708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.306905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.308386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.310342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 93/190] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.310977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.323315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.323955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.324239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.324514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] PATH_INFO: `/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.325678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] RBAC: Authorizing `identity:get_project(project_id=0a850f2d6c8b4c309175f5ab4d3817c0)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.327730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.329389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 98/191] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/projects/0a850f2d6c8b4c309175f5ab4d3817c0 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.330175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.410831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1ad2-cf1d-4ab8-b760-2339142b4d4d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.413385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.438896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc8da-7dd9-47b0-8a88-e3f9afd79803 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.446905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.447552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.447838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.448120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.454765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.461802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.463303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.464947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 94/192] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.465712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.474225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.474914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.475202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.475511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.476735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.478286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.484482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 99/193] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.484482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.569077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5f79-3766-4290-b306-19e82539b226 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.581535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fca5-5b7d-4c2c-8eb9-7029e3b24faa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.603481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.604120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.604402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.604673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] PATH_INFO: `/v3/users/31a367eff97b465091c62aa43c85cd64` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.605862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] RBAC: Authorizing `identity:get_user(user_id=31a367eff97b465091c62aa43c85cd64)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.607307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.613562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 95/194] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/users/31a367eff97b465091c62aa43c85cd64 => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.613562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.616317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.616937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.617220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.617493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.618246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:50.618552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.619917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.628502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 100/195] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.629319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.643414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.675949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.714088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4337e-d9fe-45ad-914d-3e8773c9f3ab None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.729253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a6986-4ca0-4a1a-9c89-72871ba07cc7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.746950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.747566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.747850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.748135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.749423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=1eea4fac81544f2bac9c90f3a5feb407) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:50.749732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.751195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.757156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 96/196] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/role_assignments?user.id=1eea4fac81544f2bac9c90f3a5feb407&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.757959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.769559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.770247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.770529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.770801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.771631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=0a850f2d6c8b4c309175f5ab4d3817c0, user.id=4c88a9c9720f4b08add7dd0deda4313d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:50.771944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.773452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:50.780798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 101/197] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/role_assignments?user.id=4c88a9c9720f4b08add7dd0deda4313d&scope.project.id=0a850f2d6c8b4c309175f5ab4d3817c0&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:50.781661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:50.901371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.912086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902c01-1bf9-43c9-b2ee-0ea129fb6ecc None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.935395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8d7d3-4ac4-4d62-bd96-29da1e6d0ace None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:50.953999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.954641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.954834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:50.954942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.955189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.984893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:50.984893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:50.985152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:50.985564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:50.986496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:50.988146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.015315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 102/198] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:50 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:51.016159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.042026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.044184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.161095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512c4-2f43-48f6-b931-746dfbb82729 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.169909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.179267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 97/199] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:51.180463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.209030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.209661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.209973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.210473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.256016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.308088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.308670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491e5-4e31-448f-a334-b092bb9c4bc3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.309605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.358242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.359170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.359587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.360116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.361347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.363457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.369313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 98/200] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:51.370558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.433091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 103/201] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:51.438157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.440294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.480923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874498-df86-45ed-923c-8246c0b0ac31 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.512705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.517480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.518171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.518620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.518900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.520958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.522451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.524297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 99/202] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:51.525483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.560871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d8e70-4e3a-43ea-909c-329e2f512f60 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.608900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.609520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.609808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.610136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.643199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097ff08-558f-4ee1-8610-85041ebf8835 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.682983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.688379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.689956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.691290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.692274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.693510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.693931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.695210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.696718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.699398 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 100/203] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:51.700308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.765680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:51.812526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 104/204] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:51.813885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.828002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfc254-8f5d-4580-94af-f045882de7f9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.863859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:51.864638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:51.864919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:51.865193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:51.866101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=534d5452693b4f11962cc8e55bb1b239) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:51.866411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:51.867870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:51.874183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 101/205] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/role_assignments?user.id=534d5452693b4f11962cc8e55bb1b239&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:51.875046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:51.943906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd1167-ae51-43b2-b375-4b57237102f9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:51.958081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:52.000230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:52.000868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:52.002280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:52.002577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:52.003399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, user.id=31a367eff97b465091c62aa43c85cd64) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:52.003706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:52.006544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:52.019725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 105/206] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/role_assignments?user.id=31a367eff97b465091c62aa43c85cd64&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:52.044437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418ea40-b211-4893-924d-ac00b7e5dca2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:52.097664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:52.097664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:52.097664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:52.097664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:52.099174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:52.101364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df0e7-6776-45d9-94da-4d868fdde0f1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:52.103301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 102/207] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:51 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:52.110024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:52.148801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:52.235810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e24cdb-fbe7-48da-ae6e-6206cae522b6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:52.273571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:52.274201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:52.274481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:52.274755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:52.275650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=534d5452693b4f11962cc8e55bb1b239) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:52.275964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:52.277413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a5047-55e0-4b71-bcad-bfe8b84b1c5a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:52.285235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 106/208] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:52 2022] GET /identity/v3/role_assignments?user.id=534d5452693b4f11962cc8e55bb1b239&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=6dda70e53deb474a96e637c5fff03848 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:53.344437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e0f3-5cd8-4252-8caf-a06a75cb0cb7 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.345191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e0f3-5cd8-4252-8caf-a06a75cb0cb7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.345289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e0f3-5cd8-4252-8caf-a06a75cb0cb7 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.346759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 103/209] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:53.352051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.352840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.353341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.358361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0b9e-6acb-4d8f-8694-732fe8d68f44 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.358361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0b9e-6acb-4d8f-8694-732fe8d68f44 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.358361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0b9e-6acb-4d8f-8694-732fe8d68f44 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.358361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 104/210] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:53.365084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.365624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.366134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.378448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:53.379550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:53.386970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:53.389391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:53.461670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 107/211] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:53.478021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 105/212] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:53.487740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:53.507834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:53.521391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:53.581104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:53.625101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586447d4-5281-4506-9d00-c8a0c3b152c2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:53.626550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49a63a-f8ca-46b8-90d4-db6986284502 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:53.661458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:53.662218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.662498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.662770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.664168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:53.665758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:53.666002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:53.666885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.667362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.667808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 108/213] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:53.668334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.673095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:53.680935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:53.686797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:53.687675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:53.688452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:53.695711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 106/214] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:53.697175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:53.768377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:53.777990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a204d-ff2e-43c1-83bb-77a707b6d616 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:53.807044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87788f5-da3b-4df3-a29e-d5c91958d6d5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:53.812745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:53.813371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.813651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.813957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.814638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:53.816072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:53.832452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:53.845908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 109/215] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:53 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:36:53.853895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:53.854940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:53.855424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:53.855902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:53.856897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:53.857968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:53.859897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:53.870043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 107/216] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:53.878998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:53.994752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6e43-e6fe-4889-94ff-c85c714378f1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.030803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.031461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.031744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.032023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] PATH_INFO: `/v3/users/6ece0ed543274cc89d3dd34d8add107e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.046242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] RBAC: Authorizing `identity:get_user(user_id=6ece0ed543274cc89d3dd34d8add107e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.047749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f766182-5704-4e4a-bf74-0d08ffef4206 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.049746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 110/217] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:53 2022] GET /identity/v3/users/6ece0ed543274cc89d3dd34d8add107e => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:54.055457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.098771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417db0a-5f45-453b-92df-5a44f9f6d5ed None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.099094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417db0a-5f45-453b-92df-5a44f9f6d5ed None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.099325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417db0a-5f45-453b-92df-5a44f9f6d5ed None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.100996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 111/218] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:54.106230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.106513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.106790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.131129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.140682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:54.142430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:54.180827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22684ca-f4e8-4442-aef6-f596925f84d3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.224219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.225319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.225766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.226439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.231532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 112/219] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:54.257291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.291200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.306005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.307703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.386757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321c266-97f5-4d01-9d1f-978845c380dc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.416233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 108/220] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:54.422115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.431295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.432166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.432954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.433400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.437563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.449393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1047de59-cb8a-45cd-9ac0-bcd5ed6a8fa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:54.457891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:54.458773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 113/221] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:54.459542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8ba5-a89d-42ec-94ff-2df80d1118c0 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.460310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8ba5-a89d-42ec-94ff-2df80d1118c0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.460310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8ba5-a89d-42ec-94ff-2df80d1118c0 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.460310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 114/222] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:54.465662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd8ba5-a89d-42ec-94ff-2df80d1118c0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.526280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.551226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321b29-aa40-4606-9a80-b77ba33bb149 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.595807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.596650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.596986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.597358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.598889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.600477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6eafe-3860-40c8-ac8c-524caa310a2e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.607882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 109/223] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:54.609366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd8ba5-a89d-42ec-94ff-2df80d1118c0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.610207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.610653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.611020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.629249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:54.630632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:54.653779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.654729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.655408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.655827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.657377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:54.657893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.660100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.672154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 115/224] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:54.673364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.715359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 110/225] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:54.723476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.755998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.801966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04542c7-572a-4e9f-9803-be6f17e73aaa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.817342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:54.837441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.838212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.838529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.838869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.839731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=6ece0ed543274cc89d3dd34d8add107e) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:54.840104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.840969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336fa8f-d569-4d6a-8a3f-d4c1d5fc439f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:54.841828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.849962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 116/226] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/role_assignments?user.id=6ece0ed543274cc89d3dd34d8add107e&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:54.849962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.917461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:54.918255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:54.918635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:54.918976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:54.922660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:54.924206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:54.926550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 111/227] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:54.944928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:54.984882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee2214-1659-4e6b-80ee-857b89688031 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.018202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:55.023978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.024899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.025392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.025895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.039009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:55.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:55.046336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:55.047143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:55.054219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 117/228] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:55.056267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:55.157774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c42c-88dc-4ec2-bb44-8fd74621abd7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.218655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2573680-f233-49db-9e8e-d49efa5149af None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.230487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.231158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.231439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.231715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.234762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.236286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.242446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 112/229] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:54 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:55.244479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:55.278615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:55.301882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.302675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.302959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.303239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.304029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:55.304420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.306125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.316242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 118/230] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:55.325849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:55.390201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:55.429552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc87b-0bfd-4c6b-9218-77964a6be8a9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.475564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.476227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.476514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.476812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.478088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.479927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.481678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 119/231] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:55 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:55.487369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:55.495749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a64e9-2858-43f4-8140-ae62650264c6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.587396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.588091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.588374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.588653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/534d5452693b4f11962cc8e55bb1b239/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.595387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, user_id=534d5452693b4f11962cc8e55bb1b239, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.596858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.620386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd5a77-333a-48ae-b076-3d83201d2ff3 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.629208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 113/232] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:55 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/534d5452693b4f11962cc8e55bb1b239/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:36:55.634720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:55.660041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.660692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.661064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.661348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.662529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.664040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75427-fa2f-4ef4-ad87-8bbaeacfce4e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.665950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 120/233] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:55 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:55.671853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:55.782233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:55.811058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb23092-e853-42d4-8747-7e1c9a314583 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:55.853591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:55.854232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:55.854511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:55.854784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/534d5452693b4f11962cc8e55bb1b239/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:55.857030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, user_id=534d5452693b4f11962cc8e55bb1b239, role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:55.859072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a59ba-3899-42f0-a1e8-009cdf7d2a5b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:55.876153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 114/234] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:55 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/534d5452693b4f11962cc8e55bb1b239/roles/6dda70e53deb474a96e637c5fff03848 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:56.343597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd655f57-d5ff-419c-a8ad-4585b97a9ff0 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.343963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd655f57-d5ff-419c-a8ad-4585b97a9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.344160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd655f57-d5ff-419c-a8ad-4585b97a9ff0 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.346059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 121/235] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:56.360205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf0dd-f4bf-4476-a431-e56d94e5f159 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.360525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf0dd-f4bf-4476-a431-e56d94e5f159 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.360789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbf0dd-f4bf-4476-a431-e56d94e5f159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.383924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-17dbf0dd-f4bf-4476-a431-e56d94e5f159 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:56.385093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-17dbf0dd-f4bf-4476-a431-e56d94e5f159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:56.474652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 115/236] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:56.501416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd655f57-d5ff-419c-a8ad-4585b97a9ff0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:56.534596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7137d-8504-4bf7-b9c7-db486b86bfc8 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.535092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7137d-8504-4bf7-b9c7-db486b86bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.535162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7137d-8504-4bf7-b9c7-db486b86bfc8 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.536821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 116/237] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:56.541911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.542427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.542902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.568104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:56.569373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:56.657062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd655f57-d5ff-419c-a8ad-4585b97a9ff0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:56.687729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:56.703030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 117/238] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:56.718844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:56.720028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.720510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.721036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.735851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:56.741193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:56.742103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:36:56.742880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:56.745140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 122/239] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:56.751680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:56.768791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:56.842177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:56.906399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a0978-b78c-45e8-834f-ff81a90f14ef None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:56.931419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cf005-274d-41c7-977d-b0f4ce2f7cdd None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:56.953957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:56.955015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:56.955323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:56.955598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:56.958795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:56.960351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:56.966217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 123/240] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:56.968671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.018847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.019513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.019792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.020062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.020799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:57.021107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:57.022569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:57.045618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 118/241] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:57.051039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.076886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaba6b-e60b-4b2d-8195-ee1bde5c4578 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:57.094166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:57.122176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.122883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.123682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.124380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.129841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:57.129841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:57.150424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 124/242] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:36:56 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:36:57.172291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:57.272383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad627980-e065-49e0-941b-a64a2c157078 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:57.331860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.332824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.333240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.333659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] PATH_INFO: `/v3/users/929f95b50a4c48758b68ba78bc8b7f05` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.361122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] RBAC: Authorizing `identity:get_user(user_id=929f95b50a4c48758b68ba78bc8b7f05)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:57.362633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc7a59-c80a-41fd-88ea-ec7e19a22c49 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:57.364721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 119/243] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:57 2022] GET /identity/v3/users/929f95b50a4c48758b68ba78bc8b7f05 => generated 297 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:57.370539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.482168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a88d6-d984-4940-aa65-88ae8afc8e23 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:57.517677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.518411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.518614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.518890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.551155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:57.592329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:57.593883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:57.627461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb414-034c-4914-9372-2ea1f6d63fde None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.627954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb414-034c-4914-9372-2ea1f6d63fde None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.628058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb414-034c-4914-9372-2ea1f6d63fde None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.629591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 120/244] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:57.635966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.636272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.636585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.665609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:57.666913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:57.715754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 125/245] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:57.721600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.759358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 121/246] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:57.797036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.811035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:57.871839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:57.875644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1a5e-4f4e-4061-b55f-89e3f2dc53ab None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:57.927848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.929319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.929610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.929913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:57.932056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:57.933943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:57.937529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 126/247] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:57 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:57.944454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:57.946285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a59c60-74ba-4d28-bd91-fcd7d91492fb None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:57.995810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:57.996528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:57.996811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:57.997516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.011795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:58.018391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:58.019278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:36:58.020056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:58.020640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 122/248] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:58.026454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.074595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9eb51-5f3c-4921-a514-c5f61a915d62 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.115053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:58.130694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.131674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.132151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.132599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.133573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=929f95b50a4c48758b68ba78bc8b7f05) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:58.134083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:58.135711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:58.141936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 127/249] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:36:57 2022] GET /identity/v3/role_assignments?user.id=929f95b50a4c48758b68ba78bc8b7f05&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:58.152290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe2d5-6cf5-4b13-9d99-904c7fa5a5de None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.162319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:58.194271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.194957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.195235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.195551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.196375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:58.196682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:58.202186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fe26a-9480-4d89-bc54-83746a82625b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:58.212302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 123/250] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:58.218863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.233481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778567f-80dd-4440-8333-b7038ec4fbd9 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.233795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778567f-80dd-4440-8333-b7038ec4fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.234072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778567f-80dd-4440-8333-b7038ec4fbd9 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.235631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 124/251] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:36:58.241327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.241673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.242005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.262457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:58.263939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:58.371050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 125/252] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:36:58.391547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.491081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aa799-de7e-4e4d-b5ab-9c97d818b94f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.558496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.559143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.559413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.559687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] PATH_INFO: `/v3/users/6ece0ed543274cc89d3dd34d8add107e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.561006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] RBAC: Authorizing `identity:get_user(user_id=6ece0ed543274cc89d3dd34d8add107e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:58.563733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:58.565615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 128/253] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/users/6ece0ed543274cc89d3dd34d8add107e => generated 297 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:58.572401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.600895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:58.607673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57909a65-ce23-4233-bb6f-ee652dfef539 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.674745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:58.700262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.702243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.702893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.704254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.725810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-694dca8a-7795-4f76-9b0f-ab9998258572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:58.731769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:58.732900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:36:58.733677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:36:58.735611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 126/254] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:36:58.740358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976bbe-2ebf-42dd-be6f-f170559199ed None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.742077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc32fa1-622e-419f-9f74-1603b8d695cf None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.742516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc32fa1-622e-419f-9f74-1603b8d695cf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.742959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc32fa1-622e-419f-9f74-1603b8d695cf None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.744776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 127/255] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:36:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:36:58.745991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc32fa1-622e-419f-9f74-1603b8d695cf None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.787567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.788254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.788562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.788863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.790088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:58.791574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b0edb-dfff-45d4-a70a-e056f9ea3a16 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:58.797843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 129/256] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:58.797843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.797843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.797843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.816417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:36:58.817559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:36:58.870191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc32fa1-622e-419f-9f74-1603b8d695cf None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:58.889475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 130/257] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:36:58.894417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.909164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:58.910313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:58.910834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:58.911402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:58.912330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:58.912845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:58.914592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:58.923754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 128/258] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:58.929680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:58.953995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:58.981075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.022504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25968966-9953-46fd-abd5-11cddc538ab8 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.041279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240d761-29b1-4a28-b3a6-7133ecc3332c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.058922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.059707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.060011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.060320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/6ece0ed543274cc89d3dd34d8add107e/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.062354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, user_id=6ece0ed543274cc89d3dd34d8add107e, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:59.063953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:59.078805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 131/259] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:36:58 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/6ece0ed543274cc89d3dd34d8add107e/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:36:59.080030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.126896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.127578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.127863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.128142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.135118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:59.136633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:59.138939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 129/260] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:58 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:59.144950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.250979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03509e-12cf-46bb-a994-b449d3217b25 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.278353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.289974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.290834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.291208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.291522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.305155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:36:59.311365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:36:59.312420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:36:59.313806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 132/261] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:36:59.319982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.405847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.423585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ccea-b943-4398-ba1b-d3ef184618d7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.434257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1d32e-931e-4de2-bbb3-916a7e608064 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.473848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.473848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.473848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.473848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.474231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:36:59.474231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:59.474574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:59.484784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 133/262] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:36:59.490684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.495677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.496557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.497042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.497518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.594264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.658592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:59.663270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:59.683104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616c63-5c72-4ce5-b652-9a8e52c92b92 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.758577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.759307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.759598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.759883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] PATH_INFO: `/v3/users/534d5452693b4f11962cc8e55bb1b239` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:36:59.761309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] RBAC: Authorizing `identity:get_user(user_id=534d5452693b4f11962cc8e55bb1b239)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:36:59.763069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:36:59.765878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 134/263] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/users/534d5452693b4f11962cc8e55bb1b239 => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:36:59.774906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.831913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.864626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 130/264] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:36:59.870360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:36:59.936932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf8d6e-6f7e-4f94-98c4-3f11a30fa562 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:36:59.958166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:36:59.996877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:36:59.997765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:36:59.998078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:36:59.998381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.134484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d37fb2-1a37-4863-8360-879b0fe375e9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:00.150918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:00.152475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:00.200244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:00.200984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.201263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.201534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.202794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:00.204429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:00.206866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 131/265] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:00.212422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:00.322950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:00.397450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 135/266] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:00.403441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:00.413561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779395e-47f2-4acb-921c-3fd995f09f5a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:00.434167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:00.452356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:00.452987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.453268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.453541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.454372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=534d5452693b4f11962cc8e55bb1b239) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:00.454680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:00.456117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:00.465955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 132/267] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:00 2022] GET /identity/v3/role_assignments?user.id=534d5452693b4f11962cc8e55bb1b239&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:00.621540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484a248-63ad-4e65-99de-940e5e8abd1c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:00.670987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:00.671930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.672344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.672802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.674290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:00.675736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453db07-ca8f-43a4-ad1b-08e98b08ea47 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:00.677767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 136/268] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:00 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:00.683665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:00.787457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8a19e-8c0b-469e-9038-348dbe475bf2 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.787741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8a19e-8c0b-469e-9038-348dbe475bf2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.788020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8a19e-8c0b-469e-9038-348dbe475bf2 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.793864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 137/269] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:00.796254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.796549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.796830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.817441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:00.818684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:00.826923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ef7f7-3583-4db9-b7d4-6fc3d641e229 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:00.870227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:00.899484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:00.900126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.900409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.900684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.902722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=534d5452693b4f11962cc8e55bb1b239) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:00.903056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:00.905114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaf299-497c-492b-ac0b-a2f559c5d556 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:00.920570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 133/270] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:00 2022] GET /identity/v3/role_assignments?user.id=534d5452693b4f11962cc8e55bb1b239&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=6dda70e53deb474a96e637c5fff03848 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:00.922057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615266f7-4bd8-471b-967c-d5b544f1a1d6 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.922290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615266f7-4bd8-471b-967c-d5b544f1a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.922566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615266f7-4bd8-471b-967c-d5b544f1a1d6 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.923937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 134/271] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:00.930011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999de3f-ea36-4f47-a2e2-4b6d4b485911 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:00.930299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999de3f-ea36-4f47-a2e2-4b6d4b485911 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:00.930558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999de3f-ea36-4f47-a2e2-4b6d4b485911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:00.943855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 138/272] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:00.960354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3999de3f-ea36-4f47-a2e2-4b6d4b485911 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:00.961614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3999de3f-ea36-4f47-a2e2-4b6d4b485911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:00.990840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.005649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.082423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 135/273] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:01.083918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ab1cc-3594-4dd7-b5b6-957d7700855b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.086235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ab1cc-3594-4dd7-b5b6-957d7700855b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.086526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ab1cc-3594-4dd7-b5b6-957d7700855b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.088083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 136/274] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:01.096692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.096973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.098054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.121184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:01.124251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:01.151105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cf0b-1cd4-4992-9933-984f2a8592cd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.196457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.197360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.197643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.199859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.209803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:01.215096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:01.215969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:01.217271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 139/275] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:01.220396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 137/276] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:01.221129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.229025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.279613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.304425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.339576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71727ca0-efca-4bb3-b095-05d8f50a5f80 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.342973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704418f-e072-4d68-8ff3-6ba230f78b15 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.376201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.377146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.377605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.378093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.380844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.381470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.381756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.382058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.382787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:01.383094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.384486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.390243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:01.394068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 138/277] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.394801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:01.400360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:01.401283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 140/278] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:01.402681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.499669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510845c5-2230-4690-b60c-cf2f86211c38 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.506383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.512804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77460b6e-accb-4c55-8184-76d5739522be None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.535295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.535936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.536215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.536485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.537730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.539301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.541165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 139/279] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.541757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.548917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.555202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.555933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.556223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.556499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] PATH_INFO: `/v3/users/e0a1df89d3b34c78a096dda90ab794a9` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.570719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] RBAC: Authorizing `identity:get_user(user_id=e0a1df89d3b34c78a096dda90ab794a9)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.572184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.578887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 141/280] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/users/e0a1df89d3b34c78a096dda90ab794a9 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.578887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.658254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892376d-e966-43e8-9dea-c2c9afcdf328 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.678842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe89832-f568-433c-831c-2a9cbb2d90ba None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.693812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.694509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.694790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.695073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.695768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:01.696073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.697434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.706158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 140/281] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.711374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.714805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.715427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.715710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.715987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.717204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.718684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.757930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 142/282] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:01 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:01.757930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.758185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.807284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:01.864645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144cb3-0456-403a-8a17-354421bd4126 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.882930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66d6c-90ab-4687-9d72-b82df390bcb9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:01.901739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.902401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.902684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.902958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.904281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, user.id=e0a1df89d3b34c78a096dda90ab794a9, scope.system=all) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:01.905044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.907017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d0019-0711-4762-8a30-a1ea1f5fbd41 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.912931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 141/283] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/role_assignments?user.id=e0a1df89d3b34c78a096dda90ab794a9&scope.system=all&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.913722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aaa38-e193-4a47-b212-a43c297ce9b5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.914045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aaa38-e193-4a47-b212-a43c297ce9b5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.914322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aaa38-e193-4a47-b212-a43c297ce9b5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.915592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 142/284] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:01.922150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.922452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.922725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.929154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:01.930566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:01.930566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:01.930566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] PATH_INFO: `/v3/users/929f95b50a4c48758b68ba78bc8b7f05` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:01.931973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] RBAC: Authorizing `identity:get_user(user_id=929f95b50a4c48758b68ba78bc8b7f05)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:01.934645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:01.936218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 143/285] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/users/929f95b50a4c48758b68ba78bc8b7f05 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:01.943153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:01.949282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:01.950760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:02.043796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 143/286] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:02.073618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:02.081776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2074-926a-4341-8bdb-71a35b86b519 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.098361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:02.126227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:02.137406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.138974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.139460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.139995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.141502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:02.143760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:02.146758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 144/287] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:01 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:02.151993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:02.246650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c46c03-a0de-4cb6-8fb1-39d559083fbf None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.265328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874d5a-d813-48ba-87ba-369722284a95 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.293733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.295535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.295888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.296163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.304008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.304682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.304967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.305244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/929f95b50a4c48758b68ba78bc8b7f05/roles/a4543dd62d2744a08bf092cf6c7cce33` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.307846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, user_id=929f95b50a4c48758b68ba78bc8b7f05, role_id=a4543dd62d2744a08bf092cf6c7cce33)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:02.309458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:02.310359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:02.316661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:02.317788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:02.319171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:02.322506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 144/288] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:02.327758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 145/289] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:02 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/users/929f95b50a4c48758b68ba78bc8b7f05/roles/a4543dd62d2744a08bf092cf6c7cce33 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:37:02.328805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:02.470829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:02.488258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c363ee-eb4a-4158-93d2-ac906f887f2d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.543755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.544974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.545302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.545632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.546334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:02.546640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:02.548025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:02.561102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 145/290] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:02.567216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:02.616068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:02.691116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5134-5e73-4253-b26d-d07b8645f1dd None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.733901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.734808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.735288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.735705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] PATH_INFO: `/v3/users/6ece0ed543274cc89d3dd34d8add107e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.737066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] RBAC: Authorizing `identity:get_user(user_id=6ece0ed543274cc89d3dd34d8add107e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:02.738696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:02.741475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 146/291] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:02 2022] GET /identity/v3/users/6ece0ed543274cc89d3dd34d8add107e => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:02.746132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:02.895929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745491e4-d9c1-46bd-b135-2a110c6a202d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:02.947858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:02.948475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:02.948756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:02.949027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:02.983263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:03.033070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:03.034546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:03.158916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 146/292] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:03.164565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:03.277776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95674-ff76-46d5-b7ba-45c1ba46742f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:03.293263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:03.327258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:03.327912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:03.328192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:03.328467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:03.329638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:03.331183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fd382-51b1-40b3-8a38-4fcc2714a372 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:03.337380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 147/293] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:03 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:03.338676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:03.420385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:03.482929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6260-d776-43c8-a024-aa7ea7118f85 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:03.536339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:03.536964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:03.537247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:03.537515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:03.538431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=6ece0ed543274cc89d3dd34d8add107e) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:03.538742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:03.540161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278801d7-bb77-449e-8355-7b17d8555e22 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:03.550457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 147/294] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:03 2022] GET /identity/v3/role_assignments?user.id=6ece0ed543274cc89d3dd34d8add107e&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:04.253650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dd06e-880e-4599-9a54-5f78776a985b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.254186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dd06e-880e-4599-9a54-5f78776a985b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.254262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dd06e-880e-4599-9a54-5f78776a985b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.255578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 148/295] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:04.261122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbee5f-37ac-4a4a-9d46-bb71ed8052ea None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.261544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbee5f-37ac-4a4a-9d46-bb71ed8052ea None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.261751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbee5f-37ac-4a4a-9d46-bb71ed8052ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.280698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-dfdbee5f-37ac-4a4a-9d46-bb71ed8052ea None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:04.281915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-dfdbee5f-37ac-4a4a-9d46-bb71ed8052ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:04.352812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 148/296] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:04.376015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902dd06e-880e-4599-9a54-5f78776a985b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:04.455791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d156-8db9-4e2f-bdba-63c4a878f901 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.456107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d156-8db9-4e2f-bdba-63c4a878f901 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.456356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d156-8db9-4e2f-bdba-63c4a878f901 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.457999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 149/297] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:04.464194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2aad5-a494-4439-9f34-22d9551f398a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.464478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2aad5-a494-4439-9f34-22d9551f398a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.464752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2aad5-a494-4439-9f34-22d9551f398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.488370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f8a2aad5-a494-4439-9f34-22d9551f398a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:04.489674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8a2aad5-a494-4439-9f34-22d9551f398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:04.490195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902dd06e-880e-4599-9a54-5f78776a985b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:04.532089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:04.532795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.533117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.533409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.534627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:04.536158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4884cb-a37b-443b-af28-5515f1792c02 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:04.538053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 149/298] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:04.543292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4681423-7880-4f87-ae4c-fd794da37c83 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.543626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4681423-7880-4f87-ae4c-fd794da37c83 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.544529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:04.545212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4681423-7880-4f87-ae4c-fd794da37c83 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.547519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 150/299] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:04.548244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4681423-7880-4f87-ae4c-fd794da37c83 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:04.574820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 150/300] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:04.576566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.576854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.577129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.595527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:04.596911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:04.606988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:04.665311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 151/301] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:04.665922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:04.684010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4681423-7880-4f87-ae4c-fd794da37c83 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:04.727287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:04.727903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.728192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.728455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] PATH_INFO: `/v3/groups` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.729108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:04.730630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:04.748433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 151/302] 10.222.0.66 () {60 vars in 1134 bytes} [Fri Jul 29 12:37:04 2022] POST /identity/v3/groups => generated 229 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:04.750002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:04.808219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2aa3-3449-4431-b81f-535054a7fe6f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:04.820611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:04.860261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19443-f565-4a18-baa4-11091251167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:04.861163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19443-f565-4a18-baa4-11091251167c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.861569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19443-f565-4a18-baa4-11091251167c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.862719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19443-f565-4a18-baa4-11091251167c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.875504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19443-f565-4a18-baa4-11091251167c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-08f19443-f565-4a18-baa4-11091251167c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:04.880704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:04.881857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:04.882859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 152/303] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:04.883629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866427aa-0044-41f9-99ff-c8429706f413 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.883629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866427aa-0044-41f9-99ff-c8429706f413 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.883761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866427aa-0044-41f9-99ff-c8429706f413 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.885789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 153/304] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:04.888171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866427aa-0044-41f9-99ff-c8429706f413 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:04.900747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82082f7b-1bed-4016-a16a-829c7439a468 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:04.904559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:04.943758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:04.944235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.944539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.944853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.957545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-596b9738-a49b-41a4-a6c0-a6cfc778bdaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:04.962634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:04.963518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:04.964283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 152/305] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:04.964996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff2ad-3502-4507-b300-732161705258 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:04.965490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff2ad-3502-4507-b300-732161705258 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:04.966237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff2ad-3502-4507-b300-732161705258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:04.984346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4ccff2ad-3502-4507-b300-732161705258 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:04.985599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4ccff2ad-3502-4507-b300-732161705258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:05.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866427aa-0044-41f9-99ff-c8429706f413 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.058400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 153/306] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:05.058400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccff2ad-3502-4507-b300-732161705258 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.065837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.065837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.065837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.065837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.066205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:05.066205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.066205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.074717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 154/307] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.083341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.119655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.170516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.179075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccff2ad-3502-4507-b300-732161705258 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.197192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cb2b7-1fca-4de4-9748-65c43e90f2e0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.216978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.217605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.217905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.218183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.218934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:05.219240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.220920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.234639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 154/308] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.234639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.241600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.242290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.242569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.242843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] PATH_INFO: `/v3/users/e0a1df89d3b34c78a096dda90ab794a9` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.244073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] RBAC: Authorizing `identity:get_user(user_id=e0a1df89d3b34c78a096dda90ab794a9)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.245533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.247495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 155/309] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/users/e0a1df89d3b34c78a096dda90ab794a9 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.248082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.344052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf57b0-f99d-40c4-8c01-dab705591eb0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.350813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.358698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d48b20-087b-4a80-adfe-a75f677f78e6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.381569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.382255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.382536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.382812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.393491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.393875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:05.394165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.394449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.394726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] PATH_INFO: `/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:05.399176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:05.400037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:05.400799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 155/310] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:05.401436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.412347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] RBAC: Authorizing `identity:get_user(user_id=3f65a0c3e5274e9c891f900ba5c9ff9b)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.414238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.415726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 156/311] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.418659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.425604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.519594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b41f5-9b80-4915-8e74-e3a921a104a4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.524544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a15649-7931-485e-813f-47bd097d0193 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.558827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.559617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.560427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.560792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] PATH_INFO: `/v3/system/users/e0a1df89d3b34c78a096dda90ab794a9/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.562749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e0a1df89d3b34c78a096dda90ab794a9, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.562919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.563544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.563861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.564135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.564227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.564857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:05.565159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.566534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.582502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 157/313] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.582502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.585339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 156/313] 10.222.0.66 () {58 vars in 1334 bytes} [Fri Jul 29 12:37:05 2022] PUT /identity/v3/system/users/e0a1df89d3b34c78a096dda90ab794a9/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:05.587317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.610997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.686721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:05.715114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140e76c-262f-42ee-9086-a3e328eb8b90 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.737202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24831f22-4295-4770-a55a-a15220e997ff None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.756284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.757420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.757708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.758025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] PATH_INFO: `/v3/users/929f95b50a4c48758b68ba78bc8b7f05` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.759210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=929f95b50a4c48758b68ba78bc8b7f05)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.760748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.762710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 157/314] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/users/929f95b50a4c48758b68ba78bc8b7f05 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.768929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:05.788854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.789402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.789682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.793921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.793921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, user.id=3f65a0c3e5274e9c891f900ba5c9ff9b, scope.system=all) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:05.793921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:05.793921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:05.802436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 158/315] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/role_assignments?user.id=3f65a0c3e5274e9c891f900ba5c9ff9b&scope.system=all&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:05.893390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559cd5-3599-4141-90aa-fc365d1ee5d1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:05.936511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:05.937287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:05.937662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:05.938087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:05.960501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:06.019409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:06.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:06.126828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 158/316] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:06.134925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:06.219755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:06.239911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a6d22-56eb-4bd6-9cfa-d40d94ca9644 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:06.274307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:06.274958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:06.275244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:06.275520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:06.276693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:06.278173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18581aad-8610-43a0-a753-b1f82c731bcd None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:06.285018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 159/317] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:06 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:06.286902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:06.336701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:06.390252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f4e2-ce45-4e36-9075-2482c0840cf6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:06.430717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:06.431337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:06.431616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:06.431887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:06.432679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e, user.id=929f95b50a4c48758b68ba78bc8b7f05) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:06.432984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:06.434452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b482e-e9c8-4f67-9ee7-45f4e252e11f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:06.441339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 159/318] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:06 2022] GET /identity/v3/role_assignments?user.id=929f95b50a4c48758b68ba78bc8b7f05&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:06.817363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf4596-5672-4ea6-b95b-c62cdb240c3a None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:06.817884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf4596-5672-4ea6-b95b-c62cdb240c3a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:06.817963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf4596-5672-4ea6-b95b-c62cdb240c3a None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:06.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 160/319] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:06.824333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:06.824628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:06.824905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:06.843918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:06.845373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:06.911238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 160/320] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:06.932632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf4596-5672-4ea6-b95b-c62cdb240c3a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:07.045536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf4596-5672-4ea6-b95b-c62cdb240c3a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:07.079975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:07.080548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.080832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.081110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.082272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:07.083713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c499-0305-42cd-8700-83112b295a6a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:07.085296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 161/321] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:07.090938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:07.143315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:07.200754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc8f94-9711-4e3d-94bf-91ad01f29b2b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:07.237104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:07.237728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.238033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.238312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] PATH_INFO: `/v3/groups` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.238986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:07.240438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc2f8b-ffbf-40fc-8fb8-bbe083c83bc0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:07.260396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 161/322] 10.222.0.66 () {60 vars in 1134 bytes} [Fri Jul 29 12:37:07 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:37:07.709489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2597d7-fd8c-4a14-ba18-134a065b49ba None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.709489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2597d7-fd8c-4a14-ba18-134a065b49ba None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.709489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2597d7-fd8c-4a14-ba18-134a065b49ba None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.710305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 162/323] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:07.714624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854b7cc-4089-4601-a37d-c75c985f7a21 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.715070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854b7cc-4089-4601-a37d-c75c985f7a21 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.715237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854b7cc-4089-4601-a37d-c75c985f7a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.738111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4854b7cc-4089-4601-a37d-c75c985f7a21 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:07.739606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4854b7cc-4089-4601-a37d-c75c985f7a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:07.817688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 162/324] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:07.842065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2597d7-fd8c-4a14-ba18-134a065b49ba None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:07.862055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:07.933597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8aa39-243b-4854-b3d8-62e591a84149 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.933937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8aa39-243b-4854-b3d8-62e591a84149 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.934184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8aa39-243b-4854-b3d8-62e591a84149 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.936243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 163/325] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:07.941278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:07.941561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:07.941849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:07.965045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:07.966952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:07.978825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2597d7-fd8c-4a14-ba18-134a065b49ba None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.009960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:08.037459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.038158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.038437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.038710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.058221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:08.063891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:08.064779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:08.065541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:08.069975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 163/326] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:08.070593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 164/327] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:08.071400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.093786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.185628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:08.235647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41a896-876d-4b41-8073-ca2f955079c5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.263885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc0e6e-14ed-463d-b63f-f1a82eb45877 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.305590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.306242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.306540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.306772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.307499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:08.307802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:08.309179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:08.322272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 164/328] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:08.329750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.330425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.330701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.330969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.338563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.355956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:08.361404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:08.362313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:08.363082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:08.363694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 165/329] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:08.371349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.418063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:08.524746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55c9fa-a17b-4be7-b204-d63b5ba5b006 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.561361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283b4d8-922c-4c8a-8f58-afd67347b70d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.590424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.591613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.591891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.592158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] PATH_INFO: `/v3/users/e0a1df89d3b34c78a096dda90ab794a9` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.593502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] RBAC: Authorizing `identity:get_user(user_id=e0a1df89d3b34c78a096dda90ab794a9)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:08.595239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:08.598566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 165/330] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/users/e0a1df89d3b34c78a096dda90ab794a9 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:08.612651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.622956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:08.638555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.639174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.639451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.639723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.640450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:08.640753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:08.642155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:08.657064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 166/331] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:08.663817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:08.789676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e729-bffb-435b-91ce-65d981ae342b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.807922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:08.841618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662e47f-790c-4d37-84dc-66e3eb09b922 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:08.845926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.846675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.847080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.847449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.848348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] RBAC: Adding query filter params (role.id=2c1eb45dfce946f99346c70e0f0906ae, user.id=e0a1df89d3b34c78a096dda90ab794a9, scope.system=all) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:08.848758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:08.850388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:08.869596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 166/332] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/role_assignments?user.id=e0a1df89d3b34c78a096dda90ab794a9&scope.system=all&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 510 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:08.914155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:08.915065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:08.915493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:08.915905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] PATH_INFO: `/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:08.922818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] RBAC: Authorizing `identity:get_user(user_id=3f65a0c3e5274e9c891f900ba5c9ff9b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:08.924570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72bc5f-f831-4b3b-9f72-793ee64e0583 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:08.926841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 167/333] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:08 2022] GET /identity/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:08.932962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:09.012226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:09.059848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94ea5c-b20f-43bb-90f5-1f2313a9500c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:09.097350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:09.098334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:09.098756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:09.099191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] PATH_INFO: `/v3/system/users/3f65a0c3e5274e9c891f900ba5c9ff9b/roles/a4543dd62d2744a08bf092cf6c7cce33` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:09.100812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3f65a0c3e5274e9c891f900ba5c9ff9b, role_id=a4543dd62d2744a08bf092cf6c7cce33)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:09.102464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d271-146c-427d-8733-b5c778a7e45a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:09.113496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 167/334] 10.222.0.66 () {58 vars in 1334 bytes} [Fri Jul 29 12:37:08 2022] PUT /identity/v3/system/users/3f65a0c3e5274e9c891f900ba5c9ff9b/roles/a4543dd62d2744a08bf092cf6c7cce33 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:10.065980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a855e0-b784-4866-8c47-f975365c35fa None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.066833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a855e0-b784-4866-8c47-f975365c35fa None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.067253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a855e0-b784-4866-8c47-f975365c35fa None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.069032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 168/335] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:10.074202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.075203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.075681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.099810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:10.101389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:10.183877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 168/336] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:10.227189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a855e0-b784-4866-8c47-f975365c35fa None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:10.355164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a855e0-b784-4866-8c47-f975365c35fa None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:10.401307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:10.402357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.402864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.403407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.420676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-bd554121-0e12-4a45-a77c-daadbf28b6d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:10.426802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:10.427723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:10.428537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:10.431280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 169/337] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:10.437381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:10.456784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed99a4-e5d9-4d6d-a0d3-3732c04f7179 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.457119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed99a4-e5d9-4d6d-a0d3-3732c04f7179 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.457339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed99a4-e5d9-4d6d-a0d3-3732c04f7179 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.458863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 170/338] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:10.464582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.464870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.465155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.503490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:10.504717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:10.518163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:10.576263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:10.661716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e51042-9736-47ba-a4ec-bf28b02d2278 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:10.681139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 171/339] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:10.709442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:10.710147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.710433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.710711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.711474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:10.711784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:10.713185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:10.722808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 169/340] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:10.728375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:10.734529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:10.878561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1626e1-39ce-498d-a5b8-78e504fd3faf None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:10.933407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:10.934009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:10.935887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:10.936161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:10.970065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:10.999675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7121f7-3d32-4748-8afa-c71bfc24b077 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.028649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.057257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.057949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.058233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.058511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.077459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:11.078974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f634-aba7-4d01-a5f0-cb0eb47398e0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:11.084798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2ba1f9d5-1f45-4ba7-84ee-e335e8e573e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:11.095460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:11.096371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:11.097190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:11.098674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 170/341] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:11.099877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65d3e0-d655-40f5-b529-ee1b6964f38e None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.101361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65d3e0-d655-40f5-b529-ee1b6964f38e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.101655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65d3e0-d655-40f5-b529-ee1b6964f38e None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.103710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 171/342] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:11.104407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65d3e0-d655-40f5-b529-ee1b6964f38e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.271608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 172/343] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:11.276076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.276364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.276641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.307769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65d3e0-d655-40f5-b529-ee1b6964f38e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.310827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:11.312019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:11.349628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.350291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.350574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.350851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.351650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:11.351957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:11.353411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:11.362474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 172/344] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:11.363301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.402190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.411664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.450375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 173/345] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:11.451250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.499374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b3ef2-2079-491d-80ed-770b765f24b0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.553333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.553961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.554249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.554526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.555723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:11.563007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:11.564802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 173/346] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:11.565573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.585764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456e32-b824-470b-8b0b-0a1e201c4b75 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.638675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.639761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.640256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.640737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.694298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.731035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:11.732954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:11.765904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31467a86-bc8f-4d9f-8a3a-7f00e1ac75f3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.781911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.841311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.842013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:11.842130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 174/347] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:11.842296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:11.842578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:11.842962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.863684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:11.871290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:11.872345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:11.873185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:11.877364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 174/348] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:11.878274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:11.921738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:11.957181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8d578-c01b-46b6-8c98-e7d47b9047f2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:11.998966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:11.999711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.000053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.000392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.001386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f5404-7aaa-43a5-9be0-011dae31c1c7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.014196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.015668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a3211-240c-469f-9f88-8cf398c0f3a1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.017992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 175/349] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.019191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4637c-d1a0-41ac-b4e0-51aa18d15791 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.019473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4637c-d1a0-41ac-b4e0-51aa18d15791 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.019743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4637c-d1a0-41ac-b4e0-51aa18d15791 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.020947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 176/350] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:12.021740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4637c-d1a0-41ac-b4e0-51aa18d15791 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.058381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.059073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.059575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.059883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.061161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.062795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.064928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 175/351] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:11 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.065839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.115296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.147194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4637c-d1a0-41ac-b4e0-51aa18d15791 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.191341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b819c8-506f-4846-873a-0311287354a8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.193853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.196598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.196598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.196598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.196598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:12.196598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.197444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943c175-9077-414c-b16d-ac405bf48ea3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.200247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.208762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 177/352] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.210054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70147a1-afb2-45c7-8a2d-ecebe0f8aff0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.210854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70147a1-afb2-45c7-8a2d-ecebe0f8aff0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.210854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70147a1-afb2-45c7-8a2d-ecebe0f8aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.231084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a70147a1-afb2-45c7-8a2d-ecebe0f8aff0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:12.232891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a70147a1-afb2-45c7-8a2d-ecebe0f8aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:12.244328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.244950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.245228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.245596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.246320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:12.246625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.249899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.255633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 176/353] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.256497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.322654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 178/354] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:12.323986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b721-19d3-4e44-8c1d-4a7b77974c95 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.324273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b721-19d3-4e44-8c1d-4a7b77974c95 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.324544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b721-19d3-4e44-8c1d-4a7b77974c95 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.325998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 179/355] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:12.327099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a884b-9d8b-45e6-b78c-3dfc9ad98b5a None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.327382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a884b-9d8b-45e6-b78c-3dfc9ad98b5a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.327654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a884b-9d8b-45e6-b78c-3dfc9ad98b5a None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.333175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 180/356] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:12.333175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a884b-9d8b-45e6-b78c-3dfc9ad98b5a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.406338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9c658-6655-4060-be33-4daeffee7314 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.408591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.461595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.462288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.462570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.462901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.464054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.464164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a884b-9d8b-45e6-b78c-3dfc9ad98b5a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.465586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba902a4-79ae-4d31-a2de-87b689866d31 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.468760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 177/357] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.469426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07007-e068-454c-9a28-e59114a7bffa None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.469714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07007-e068-454c-9a28-e59114a7bffa None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.470023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07007-e068-454c-9a28-e59114a7bffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.489239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a1d07007-e068-454c-9a28-e59114a7bffa None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:12.490615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a1d07007-e068-454c-9a28-e59114a7bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:12.506376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.513272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.514240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.514522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.514791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.583746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 178/358] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:12.586503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.587103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.587740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.600735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.601583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.616412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:12.617615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:12.700654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 179/359] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:12.706738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.741092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 181/360] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:12.742917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.749013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.785049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:12.840153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e763f0-4805-427f-8c62-6d72c69ae5a4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.884411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256c978-8caa-4a01-8328-e85f8c7bb47c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:12.895153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.895827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.896109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.896491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.913271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:12.919380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:12.920305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:12.921079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 180/361] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:12.921723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:12.944124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:12.944751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:12.945032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:12.945309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:12.946165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:12.946473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:12.947921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:12.957938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 182/362] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=6dda70e53deb474a96e637c5fff03848 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:12.957938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.070207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.085515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a5b3-b939-4215-8584-36eb989490a6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.095380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.112884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d1a9b-b4f8-41bc-a035-5ac529404074 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.134072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.134739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.135005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.135275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.149039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:13.155698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:13.156611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:13.157417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 181/363] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:13.158221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.163277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.164581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.165085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.165367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.183522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:13.189999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:13.190868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 12:37:13.191647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:13.194252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 183/364] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:13.195221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.302326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707ddb-8d61-4218-8546-40785bec8b54 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.327540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f650c-acbb-4c09-9577-4a3078ebb286 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.349670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.350224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.351189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.351577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.353655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.355570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.358221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 182/365] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:13.359393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.368623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.369464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.369957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.370388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.371319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:13.371773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.373951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.383121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 184/366] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:13.384133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.385074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.403536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.493044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec6fc-ba21-4710-8618-0358c5c78ea8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.536696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.537427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.537850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.538058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.538794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:13.539155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.540712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.548394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912412-0142-47a8-a111-60ecfa3cada4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.550974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 183/367] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:13.551837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.591001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.591729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.592011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.592284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.593012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:13.593366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.595070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.607090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 185/368] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:13.608151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.643313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.664921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1613-8508-47e5-9456-1e6b2b5453d8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.701124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.703004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.703881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.704285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.704939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.706815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.708366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.715060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 184/369] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:13.715060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:13.735859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4807-b9f6-4168-a95d-cd56d1acb3b7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.773872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.774815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.775292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.775876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.830169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debb604-fa65-4409-a7df-7fb17044cd1a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:13.858138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.860113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.868173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:13.868827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:13.869112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:13.869451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:13.890915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.949754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:13.959616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:13.962484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:13.975062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 186/370] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:13.975751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.080284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d13fc0-28c9-4067-be48-f690b0be9c85 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.084826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 185/371] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:14.087603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.123136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.123681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.123964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.124238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] PATH_INFO: `/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.125426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] RBAC: Authorizing `identity:get_user(user_id=3f65a0c3e5274e9c891f900ba5c9ff9b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.126917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.128826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 187/372] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:37:13 2022] GET /identity/v3/users/3f65a0c3e5274e9c891f900ba5c9ff9b => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.129588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.166236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.178960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.221483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eafc6-d79b-4438-9c85-0f8d84650a6e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.238657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c872-a68b-4b97-9e23-941c10e97f25 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.267721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.268449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.268785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.269087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.270105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:14.270479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.272368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.275547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.276220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.276489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.276768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.278040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.279534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.280587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 186/373] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.281434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 188/374] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.281811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.282536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.416149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510647-aa55-4c5e-ab2d-cc2089c26855 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.431254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ee7a-1454-4270-b53b-02dfc806c2e2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.432336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.451054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.473251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.474324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.474324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.474513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] PATH_INFO: `/v3/projects/9e5a5cdc1deb42199be2e349eba98664` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.475740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] RBAC: Authorizing `identity:get_project(project_id=9e5a5cdc1deb42199be2e349eba98664)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.477235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.481867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 187/375] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/projects/9e5a5cdc1deb42199be2e349eba98664 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.483009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.490436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.491406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.491930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.492430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.493471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] RBAC: Adding query filter params (role.id=a4543dd62d2744a08bf092cf6c7cce33, user.id=3f65a0c3e5274e9c891f900ba5c9ff9b, scope.system=all) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:14.495845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.498216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.522262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 189/376] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/role_assignments?user.id=3f65a0c3e5274e9c891f900ba5c9ff9b&scope.system=all&role.id=a4543dd62d2744a08bf092cf6c7cce33 => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.523065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.663880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ffca-d044-4af0-881b-b7930f073a4d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.700516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c4b0-d0bf-4409-b72f-e41db0ebb5da None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:14.740498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.741172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.741452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.741730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.745849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.745849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fbc78-174f-4cf9-83f7-58f9fdafee8a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.746046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:14.746046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.746192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.746453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] PATH_INFO: `/v3/groups/fff8dd2a60094dbda67606e0cea2621d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.748076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 188/377] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.749754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0400d-6bd4-48cb-9b00-81c6c437e0bf None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.750278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0400d-6bd4-48cb-9b00-81c6c437e0bf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.750754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0400d-6bd4-48cb-9b00-81c6c437e0bf None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.753092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 189/378] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:14.754133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0400d-6bd4-48cb-9b00-81c6c437e0bf None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.755220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] RBAC: Authorizing `identity:get_group(group_id=fff8dd2a60094dbda67606e0cea2621d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:14.756876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd3af7-be18-4943-9f16-a3e2e54da535 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:14.761235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 190/379] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/groups/fff8dd2a60094dbda67606e0cea2621d => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:14.763637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:14.768155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:14.768769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:14.799168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:14.801590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:14.846229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.862527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:14.917966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 191/380] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:14.919488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:14.952505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0400d-6bd4-48cb-9b00-81c6c437e0bf None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.004987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.005633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.005971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.006252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.007063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:15.007387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:15.008902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:15.017710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 190/381] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=6dda70e53deb474a96e637c5fff03848 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:15.018608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.065230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb3e7d-ae54-4226-848c-ad111b7f8c4a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.112621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.113309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.113598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.113917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.114952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] RBAC: Adding query filter params (group.id=fff8dd2a60094dbda67606e0cea2621d, role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=9e5a5cdc1deb42199be2e349eba98664) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:15.115335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:15.116833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:15.128147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 192/382] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:14 2022] GET /identity/v3/role_assignments?group.id=fff8dd2a60094dbda67606e0cea2621d&scope.project.id=9e5a5cdc1deb42199be2e349eba98664&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:15.219972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1e072-73ea-40fe-b56b-acbf0b4de86f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.240541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:15.266329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.267444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.267964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.268518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.285467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:15.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:15.293222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:15.296134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:15.305850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 191/383] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:15.315051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.378304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:15.456327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e1b0-7c8c-494b-bbc1-fde07b8a296b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.492511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.493055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.493336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.493668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.494528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:15.494831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:15.496321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca29ca-f1c8-41c3-8304-51a0df3d63ca None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:15.509937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 193/384] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:15.512147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.555473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb65e7-0161-44d7-849c-66ba18561fe4 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.555791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb65e7-0161-44d7-849c-66ba18561fe4 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.556069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb65e7-0161-44d7-849c-66ba18561fe4 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.557533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 194/385] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:15.562945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.563228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.563500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.591649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:15.593357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:15.636449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52eae2-9c3f-4f41-9074-788721bdac2c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.689792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.690471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.690757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.691032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.693246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:15.694841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:15.700989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 192/386] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:15 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:15.703930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.756253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:15.787631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 195/387] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:15.822661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533e24-8fbe-4b3b-8631-80b951c0dd92 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:15.823050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.861973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:15.862900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:15.863778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:15.864212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:15.865692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:15.867563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:15.874231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 193/388] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:15 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:15.876302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:15.877621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:15.988972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8375-9d2c-4ea3-8ce6-65ff24ca06a4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.008500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:16.028591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.029282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.029571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.029873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.032239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:16.033743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:16.053465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 194/389] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 29 12:37:15 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:37:16.107691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0cba9-6219-40b2-84fe-ec953dc1f384 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.161562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.162333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.162614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.162891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.174576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:16.184054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:16.185736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:16.186770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 196/390] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:16.194124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:16.325639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2b39-3d14-45d6-afc3-326f5370cfcb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.363071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.363708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.363990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.365583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.366364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:16.366747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:16.368149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:16.377653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 195/391] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:16.386902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:16.487093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:16.509784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbf829-37f8-4275-a494-1968765a3889 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.549290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.550004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.550292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.550590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.551790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:16.553227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27b5a3-bd71-42d2-a75c-728233a7f2e0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:16.555275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 197/392] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:16 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:16.560483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:16.610008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:16.717011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642ff3-2a58-433c-b171-bf9b5de76b44 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.761190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.761898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.762186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.762528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.763697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:16.765138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:16.769525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 196/393] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:16 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:16.773335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006de88d-215f-429b-a0ed-8add03b1f7b3 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.773663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006de88d-215f-429b-a0ed-8add03b1f7b3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.773972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006de88d-215f-429b-a0ed-8add03b1f7b3 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.775526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:16.775654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 198/394] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:16.783162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.783462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.783747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.824278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:16.825598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:16.894191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:16.900589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d30a5f-3073-4f0d-9bb9-1191333c8525 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:16.941958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:16.942878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:16.943206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:16.943489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:16.945700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] RBAC: Authorizing `identity:create_grant(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd, group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:16.947958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:16.956876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 199/395] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:16.962365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 197/396] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 29 12:37:16 2022] PUT /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/6dda70e53deb474a96e637c5fff03848 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:37:16.992403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:17.131512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:17.162178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6c2a2-5a8b-4624-b2fe-74a9c0fd6c2d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:17.199441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:17.200645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.200928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.201205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.212299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:17.217721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:17.220093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:17.222236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 200/397] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:17.231736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:17.386690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298688-5c7b-41bc-bb95-9c5638cea06b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:17.429038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:17.429597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.429903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.430188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.430920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] RBAC: Adding query filter params (name=member) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:17.431228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:17.432686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:17.446174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 198/398] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:17.454896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:17.604281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee794-9784-45fb-bfc2-9f7c8c9210c6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:17.645229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:17.645915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.646198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.646476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.647642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:17.649122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:17.651005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 201/399] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:17 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:17.657371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:17.704906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:17.813591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9ba05-e41b-4111-ac54-6676ee287373 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:17.853288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:17.854026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.854309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.854590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.855823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:17.857846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613376d-b9cf-4ca5-8bd3-0632169cc709 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:17.859743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 199/400] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:17 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:17.865225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:17.879670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:17.917022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cb33-5b29-463c-87d6-32cc59292e74 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.917324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cb33-5b29-463c-87d6-32cc59292e74 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.917571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681cb33-5b29-463c-87d6-32cc59292e74 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.919999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 200/401] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:17.924982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8061fb-dc26-4f54-b296-11a675d30b03 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:17.925264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8061fb-dc26-4f54-b296-11a675d30b03 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:17.925540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8061fb-dc26-4f54-b296-11a675d30b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:17.945339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cd8061fb-dc26-4f54-b296-11a675d30b03 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:17.946571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cd8061fb-dc26-4f54-b296-11a675d30b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:18.026069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b3e8-d113-4581-ae87-d7b17de19b68 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.036966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 201/402] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:18.038559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3a06-eb05-4c33-9354-edb22aa3ce3b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.039047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3a06-eb05-4c33-9354-edb22aa3ce3b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.039555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3a06-eb05-4c33-9354-edb22aa3ce3b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.041250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 202/403] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:18.049844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.049844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.049844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.071863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:18.073027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:18.077255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.077970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.078256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.078532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.080492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:18.081978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:18.088907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:18.100580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 202/404] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 29 12:37:17 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:18.101537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.151924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 203/405] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:18.178830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.244398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:18.314866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82990ca-ae31-4453-b888-729e03a99174 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.346099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e037bfe-2a3d-4865-a072-2c5bb031b842 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.354696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.355352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.355632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.355969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.368788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:18.375498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:18.376584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:18.377523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 203/406] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:18.382171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.398419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.399795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.400251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.400937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.425330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:18.430466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:18.431505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:18.432365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 204/407] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:18.439230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.452052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:18.562083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1ad14-5341-4537-bf18-ce0fb951d58f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.604519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.606492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.606987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.607585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.608644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:18.609152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:18.610922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:18.620670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd014d-c559-4e98-b797-c27494fff987 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.621286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 205/408] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:18.628662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.658720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:18.677316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.678010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.678291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.678565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.680512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:18.680824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:18.684353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:18.703777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 204/409] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:18.710221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.776018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:18.834647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5333fa-9633-48e5-9fee-d69b09f8f4fa None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.895198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187c3c-b82d-47b6-b164-6aae989ac089 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:18.907527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.908196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.908467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.908761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.911479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:18.912906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:18.914540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 206/410] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:18.920455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:18.976070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:18.976735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:18.977020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:18.977295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] PATH_INFO: `/v3/groups/fff8dd2a60094dbda67606e0cea2621d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:18.982554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] RBAC: Authorizing `identity:get_group(group_id=fff8dd2a60094dbda67606e0cea2621d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:18.984076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:18.986148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 205/411] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/groups/fff8dd2a60094dbda67606e0cea2621d => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:18.992705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.047091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929eb60-d668-45f9-9852-1dd051115320 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.062487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.087910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.088560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.088847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.089162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.090884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.092394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40ed6e-1c21-4ce4-b989-deaad5fd24cd None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.094672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 207/412] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:19.095954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce1bb0-ca03-4be5-a050-edb94416349d None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.096237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce1bb0-ca03-4be5-a050-edb94416349d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.096519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce1bb0-ca03-4be5-a050-edb94416349d None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.101875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 208/413] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:19.101875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce1bb0-ca03-4be5-a050-edb94416349d None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.114617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.139118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40434-1241-46f5-8c2a-9a239648f78d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.188891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.190447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.190886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.191598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] PATH_INFO: `/v3/projects/9e5a5cdc1deb42199be2e349eba98664` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.193200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] RBAC: Authorizing `identity:get_project(project_id=9e5a5cdc1deb42199be2e349eba98664)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.195583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68187e6-f4ea-4699-885c-d181764bb33e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.198683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 206/414] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:18 2022] GET /identity/v3/projects/9e5a5cdc1deb42199be2e349eba98664 => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:19.200069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4122-3db6-4a80-b1cc-a782f0687d43 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.200780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4122-3db6-4a80-b1cc-a782f0687d43 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.201212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4122-3db6-4a80-b1cc-a782f0687d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.225962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d1bb4122-3db6-4a80-b1cc-a782f0687d43 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:19.228160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d1bb4122-3db6-4a80-b1cc-a782f0687d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:19.242186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce1bb0-ca03-4be5-a050-edb94416349d None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.284309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.285191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.285566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.286002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.288368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf7a1f30844104af5cd81aa5262e3e, group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.290000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.306293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 209/415] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 29 12:37:19 2022] PUT /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578/roles/6dda70e53deb474a96e637c5fff03848 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:19.307698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.323986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 207/416] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:19.334883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e717-6a6d-4ee2-904a-732f65a80bd0 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.335335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e717-6a6d-4ee2-904a-732f65a80bd0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.335714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e717-6a6d-4ee2-904a-732f65a80bd0 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.339395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 208/417] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:19.345810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.346193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.346589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.374697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:19.376073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:19.404628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.437318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.453798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df57491-6802-48d4-b664-73e2ffbdbe7a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.473237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 209/418] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:19.474133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.496235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.497429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.497978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.498475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] PATH_INFO: `/v3/projects/9e5a5cdc1deb42199be2e349eba98664/groups/fff8dd2a60094dbda67606e0cea2621d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.500947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e5a5cdc1deb42199be2e349eba98664, group_id=fff8dd2a60094dbda67606e0cea2621d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.502766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.518821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 210/419] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 29 12:37:19 2022] PUT /identity/v3/projects/9e5a5cdc1deb42199be2e349eba98664/groups/fff8dd2a60094dbda67606e0cea2621d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:19.520119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.599516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4442d2-a385-4dcf-9ed5-250b16d5b5d8 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.662182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86a397-4971-48e4-be66-9b6dc733d412 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.674703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.675569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.675860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.676138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.696858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:19.707990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:19.708897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:19.709685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 210/420] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:19.720036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.724494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.746100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.748694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.748694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.748694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.763830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:19.769837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:19.770736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:19.771559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:19.774323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 211/421] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:19.779948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:19.783991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.842939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ff33-9237-4d18-9069-a5446acd49d9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.883003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.883990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.884472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.884944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.885932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:19.886453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.888154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.897084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 211/422] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:19.913660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d001955-1027-4eae-8f70-56ff087381b7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:19.916750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:19.953034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:19.953658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:19.953986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:19.954263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:19.955284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:19.955609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:19.957167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:19.967191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 212/423] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:19.972611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.032376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.094452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ad3b-7c37-4f00-a9fe-1844b0ff72e8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.148679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:20.149503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.149916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.150295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.158469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05690d8f-b7a6-46a1-942c-770e6524caa7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.170795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.241144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:20.241808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.242115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.242391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.243862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:20.245329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:20.320086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.395387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:20.396848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:20.405391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 213/424] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 12:37:20.410514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.562447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 212/425] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 12:37:20.568095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.585863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.594624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ef447-296d-422d-857c-a96ce94f11a4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.663711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:20.664365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.664647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.664925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.666149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:20.667971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:20.670410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 214/426] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:20 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:20.675803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.711342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.760375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc75781-4576-411e-ac2c-875af6ecd627 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.809659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:20.810341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.810878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.811163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] PATH_INFO: `/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.813119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] RBAC: Authorizing `identity:get_project(project_id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:20.814440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f49a2-5122-40a9-b907-f29b9681d338 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.814572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6180d0-ae56-46bb-b81d-55ee43957136 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:20.817614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 213/427] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:20 2022] GET /identity/v3/projects/82bdf5ceb69f4dcd90b4d2ca2c2cf0cd => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:20.818902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a71d-2f64-4b95-9a5d-e558495de268 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.819189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a71d-2f64-4b95-9a5d-e558495de268 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.819532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a71d-2f64-4b95-9a5d-e558495de268 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.825808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 214/428] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:20.826756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a71d-2f64-4b95-9a5d-e558495de268 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.854368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:20.855119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.855465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.856345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.857573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:20.859186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864bef6-0f22-4f94-9334-8995d2953eed None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:20.861253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 215/429] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:20 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:20.862613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:20.862971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:20.863325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:20.881378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:20.882713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:20.936925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:20.960874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 216/430] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:20.961878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:20.967982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a71d-2f64-4b95-9a5d-e558495de268 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:20.998221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:21.015178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.015831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.016115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.016394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.017568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:21.019316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:21.021058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 215/431] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:20 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:21.021855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.118993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3539f-7da7-48aa-9f12-e13c5a15342f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.155742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c4b2-61ee-4b66-99c7-e9291dd1b6f6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.159642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.160463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.160920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.161337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.162321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:21.162794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:21.164499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:21.178353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 217/432] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:20 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=6dda70e53deb474a96e637c5fff03848 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:21.179605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.199158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.199859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.200150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.200425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.229760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:21.235471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:21.236443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 12:37:21.237228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:21.247062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 216/433] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:21.248020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.259233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:21.317378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:21.346895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c479a41-92d5-4bed-b760-9d914aa66e06 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.398240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.399727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.400209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.400679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.402102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:21.403168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:21.405537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:21.415112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 218/434] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=82bdf5ceb69f4dcd90b4d2ca2c2cf0cd&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:21.438325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89051c67-9718-45c1-93c2-a0b3523ba14c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.480745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.481459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.481744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.482095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.482849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:21.483181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:21.484617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05035c-b2fc-45db-acaf-df558743efa0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:21.494829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 217/435] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:21.501216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.561674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644741b-463f-4c61-b89b-a7e16a9b578b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.562213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644741b-463f-4c61-b89b-a7e16a9b578b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.562705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644741b-463f-4c61-b89b-a7e16a9b578b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.564511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 218/436] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:21.573270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.573591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.573898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.606580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:21.607720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:21.633694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f52a45-69d9-433c-878c-fc53f5733df5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.673953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.674517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.674803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.675083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.682125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:21.690738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:21.698915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 219/437] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:21.723221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.777178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:21.779170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:21.848571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cddee-9105-4e9e-bacb-d07f853f2f24 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:21.896314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:21.897346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:21.897851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:21.898307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:21.914430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:21.916657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 219/438] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:21.920006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:21.920888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:21.921672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 220/439] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:21.923511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.927978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:21.977718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.002303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.047712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfc49-b1c0-41df-901b-02a0666fda0b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.053069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222336f-be1a-4380-8d5a-853868694faa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.083776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.084426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.084708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.084982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.086189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.087629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fad93-7bbf-4803-919d-e90d56c28f2f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.089276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 220/440] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:22.089530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.090701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.090945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9d967-9645-4019-9e4e-7724efc07209 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.091039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.091229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9d967-9645-4019-9e4e-7724efc07209 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.091333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.091503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9d967-9645-4019-9e4e-7724efc07209 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.092018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:22.092323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.092852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 221/441] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:22.093742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830da79d-23af-4e98-96dd-0578a051f47b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.095635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9d967-9645-4019-9e4e-7724efc07209 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.103002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 221/442] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:22.104454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.104745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.105024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.125326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:22.127179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:22.198096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 222/443] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:22.198864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.206323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9d967-9645-4019-9e4e-7724efc07209 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.218238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.243973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.244621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.244930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.245212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.246938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.248404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.250426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 222/444] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:22.251078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.258144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.315143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e6aa2-6d52-40cb-bfb2-ad81843928ac None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.348403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9863cc-2005-4dad-86d6-1cff9e82af3d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.348706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.349327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.349611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.349927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.382956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.383630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.383939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.384221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.395087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:22.400589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:22.401634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:22.402578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 223/445] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:22.404092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.422601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.424090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.441782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.469761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.502761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581bada-dc50-405b-b02a-56a264c57f58 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.524561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 223/446] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:22.525380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.536832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.537446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.537726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.538030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.538819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=2c1eb45dfce946f99346c70e0f0906ae, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:22.539131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.540561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.546980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 224/447] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=2c1eb45dfce946f99346c70e0f0906ae => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:22.547600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.661052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0cd40-d75f-4860-85ff-89e46ca16ca6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.664117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f781d2-b461-4467-b413-0caea91c539e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.665848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.701982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.702653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.702936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.703871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] PATH_INFO: `/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.704421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] RBAC: Authorizing `identity:get_project(project_id=dcaf7a1f30844104af5cd81aa5262e3e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.704920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.705549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.705851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.706020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.706170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.706948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:22.707198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.707801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 225/448] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/projects/dcaf7a1f30844104af5cd81aa5262e3e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:22.708700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.713907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.723016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 224/449] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:22.726686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.736363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.833467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e63bb-6c42-4beb-afa7-7212819cef53 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.844427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc3a4f-97c3-435d-a5f8-1ed8f802d47d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:22.874512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.875568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.875959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.876238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] PATH_INFO: `/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.877465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] RBAC: Authorizing `identity:get_group(group_id=0be0a67a3d6c4c79bfb8e3ecfd8fa578)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.879184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.880943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 226/450] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/groups/0be0a67a3d6c4c79bfb8e3ecfd8fa578 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:22.883803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:22.884429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:22.884716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:22.884993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:22.888194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:22.935329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.966510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:22.967981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:22.978042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:22.997645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c07-135c-449a-aa1a-7809bcb55e1d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:23.032571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:23.033180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:23.033459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:23.033731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:23.034541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] RBAC: Adding query filter params (group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578, role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:23.034851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:23.036272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:23.042158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 227/451] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/role_assignments?group.id=0be0a67a3d6c4c79bfb8e3ecfd8fa578&scope.project.id=dcaf7a1f30844104af5cd81aa5262e3e&role.id=6dda70e53deb474a96e637c5fff03848 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:23.069318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 225/452] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 12:37:23.075739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:23.182320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bc3dc-6e95-4294-9e0e-f4ff067b6293 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:23.183919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:23.218847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:23.219762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:23.220048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:23.220328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] PATH_INFO: `/v3/projects/9e5a5cdc1deb42199be2e349eba98664` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:23.221511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] RBAC: Authorizing `identity:get_project(project_id=9e5a5cdc1deb42199be2e349eba98664)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:23.223417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:23.229279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 228/453] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:37:23 2022] GET /identity/v3/projects/9e5a5cdc1deb42199be2e349eba98664 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:23.239354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:23.344610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490bdf2-97d7-4f8e-999e-26ac88c1866b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:23.354695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:23.382332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:23.382972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:23.383253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:23.383524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] PATH_INFO: `/v3/groups/fff8dd2a60094dbda67606e0cea2621d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:23.384668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] RBAC: Authorizing `identity:get_group(group_id=fff8dd2a60094dbda67606e0cea2621d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:23.386145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b60db9-baa4-432e-ab98-de7f9dd89e2a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:23.387802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 226/454] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 29 12:37:23 2022] GET /identity/v3/groups/fff8dd2a60094dbda67606e0cea2621d => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:23.393841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:23.490196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251bc1-63d9-4db7-9485-8c44bed499b6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:23.521224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:23.521841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:23.522120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:23.522393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:23.523170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] RBAC: Adding query filter params (group.id=fff8dd2a60094dbda67606e0cea2621d, role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=9e5a5cdc1deb42199be2e349eba98664) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:23.523478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:23.524895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4421-7cdd-4e64-a3a1-fcd5e3623696 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:23.530961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 229/455] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 29 12:37:23 2022] GET /identity/v3/role_assignments?group.id=fff8dd2a60094dbda67606e0cea2621d&scope.project.id=9e5a5cdc1deb42199be2e349eba98664&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:25.902168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a85559-c1b4-4943-86dd-567ebd0ac45b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:25.903558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a85559-c1b4-4943-86dd-567ebd0ac45b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:25.903984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a85559-c1b4-4943-86dd-567ebd0ac45b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:25.905937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 227/456] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:25.910836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:25.911687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:25.912186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:25.936552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:25.938103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:25.975676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:26.022702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 230/457] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:26.058289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a85559-c1b4-4943-86dd-567ebd0ac45b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:26.200461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a85559-c1b4-4943-86dd-567ebd0ac45b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:26.251410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:26.252065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.252357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.252637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.264425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:26.265916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-56d7aee5-800d-4191-ab28-a999136d607a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:26.285532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:26.286470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:26.287306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:26.287306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:26.287306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:26.287306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:26.287306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 228/458] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:26.294417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:26.441395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0343c3-d043-4ba0-b2f5-d5623f0a58d0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:26.457902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731402af-ed79-4305-9e08-68bc590a2572 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.458176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731402af-ed79-4305-9e08-68bc590a2572 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.458397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731402af-ed79-4305-9e08-68bc590a2572 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.460229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 229/459] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:26.474996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697ae8e-e1b0-4563-a2b1-8e9b71279da2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.475540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697ae8e-e1b0-4563-a2b1-8e9b71279da2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.476029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697ae8e-e1b0-4563-a2b1-8e9b71279da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.481839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:26.481839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.481839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.481839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.482162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:26.482162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:26.485850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:26.495575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 231/460] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:26.508771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:26.511930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f697ae8e-e1b0-4563-a2b1-8e9b71279da2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:26.513307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f697ae8e-e1b0-4563-a2b1-8e9b71279da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:26.535752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:26.564548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:26.618500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef09288-0d0f-4555-88d4-321c1c42c16b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:26.639470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 230/461] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:26.641086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c04e1a-24d9-4da7-96ae-919cddb18391 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.641574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c04e1a-24d9-4da7-96ae-919cddb18391 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.642115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c04e1a-24d9-4da7-96ae-919cddb18391 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.643869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 231/462] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:26.648765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.649289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.649756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.668941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:26.670067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.670629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.671148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.672678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:26.674667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:26.678202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:26.679510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:26.704097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 232/463] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:26 2022] POST /identity/v3/users => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:26.714756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:26.825171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 232/464] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:26.859178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:26.875474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:26.888517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5651c-cd2f-49a8-b173-84fde5beb4d7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:26.937426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:26.938228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:26.938392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:26.939459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:26.955841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:26.957960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:26.976950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:26.978925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:26.980197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:26.980197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:26.980197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:26.980197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:26.983345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 233/465] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:26.990822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.020409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:27.038630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa4eed-eb31-4115-ba63-4fcc5fa1f3fd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.076644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.077325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.077603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.077998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.088555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.090069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d1dc961c-2cd9-4561-a1b3-4a8fcfaafdfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:27.113666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:27.115020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 233/466] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:27.115853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dbab-7248-4057-8333-63bc4f375379 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.118503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dbab-7248-4057-8333-63bc4f375379 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.118818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dbab-7248-4057-8333-63bc4f375379 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.120437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 234/467] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:27.123343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6dbab-7248-4057-8333-63bc4f375379 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.136037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a07dc-a9ca-4041-ab8e-3a98384c69bb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.170880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:27.179876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.180710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.181179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.181659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.183005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:27.183538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.185497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364f570-81b5-46ea-8f66-15dc713e1c61 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.202760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 234/468] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:27.204300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.204996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.206085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.229297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:27.231415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:27.233846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6dbab-7248-4057-8333-63bc4f375379 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.271368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.272226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.272597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.272967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.273957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:27.274418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.276527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.288912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:27.289291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 235/469] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:27.293868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.319637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 235/470] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:27.320942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.419252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec8a79-83f1-4a08-91f7-02c66031378c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.454737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.455637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.456136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.456612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.457471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.459119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.481536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 236/471] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:27 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:27.482825 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.525460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ccaf5-4cf8-43de-9e09-c5f187c97e18 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.563316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:27.581150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.582239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.584151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.584881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.587969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.588124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:27.592324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.641516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 236/472] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:27 2022] POST /identity/v3/users => generated 248 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:27.668445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18997-acb0-4d02-a830-6f61ebf8477e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.709863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.711070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.711596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.712066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.726502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.728388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-722b50df-5ead-43b3-8831-a163985dbb9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:27.745814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:27.746893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:27.747865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:27.747865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:27.747865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:27.747865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:27.750298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 237/473] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:27.759737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:27.883153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f627f76-2b21-44bb-ab24-051845773741 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:27.910875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605457d-3207-4457-96d6-5d4c49c60068 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.911166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605457d-3207-4457-96d6-5d4c49c60068 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.911455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605457d-3207-4457-96d6-5d4c49c60068 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.912815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 238/474] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:27.919597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.919885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.920156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.924306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:27.925220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:27.925793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:27.926296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:27.927174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:27.927659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:27.929559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:27.942215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:27.943544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:27.945196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 237/475] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:27.954061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:28.015651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:28.037946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 239/476] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:28.057007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:28.095437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b2076-34ec-42e2-9c0d-2cdb18723cf6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:28.107199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:28.142218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:28.143226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:28.143699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:28.144162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:28.145087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:28.146876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:28.176568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 238/477] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:27 2022] POST /identity/v3/users => generated 249 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:28.187452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d5bc4-b8d0-48c0-ae5e-952461f2be61 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:28.233649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:28.234332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:28.234617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:28.234891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:28.235644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:28.237068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:28.251244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.sql.core [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 12:37:28.251244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 12:37:28.251244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [parameters: {'id': '4f190cb4e7b74e63af0c34abb5f4dd3c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 12:37:28.251244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91820) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result.read() Jul 29 12:37:28.271983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return ref.to_dict() Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:37:28.272912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application yield resource Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise self.value Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application session.commit() Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 12:37:28.274340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.session.flush() Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application compat.raise_( Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 12:37:28.275450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:37:28.276313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result.read() Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:37:28.277115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [parameters: {'id': '4f190cb4e7b74e63af0c34abb5f4dd3c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:28.280056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 12:37:28.282027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:28.285218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 240/478] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 29 12:37:28 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:28.290820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:28.424950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:28.434093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2638ee-ac43-4e05-bb73-440bba0f3c88 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:28.471438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:28.472423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:28.472897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:28.473389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:28.485237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-c2229bb4-0eda-4ea5-b918-2624a23a0be3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:28.491312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:28.492179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 12:37:28.492959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:28.499978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 239/479] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:37:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:28.501615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:28.610528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f709e0-94ea-46ca-8494-efe1d8a3312a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:28.612438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:28.657336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:28.658371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:28.658926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:28.659405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:28.660328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:28.660853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:28.662606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450653d-dac6-4508-9769-516cdb53e480 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:28.672758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 241/480] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:29.419193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd98dc-0658-4b18-bd4d-23ce332f9f11 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:29.419673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd98dc-0658-4b18-bd4d-23ce332f9f11 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:29.419673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd98dc-0658-4b18-bd4d-23ce332f9f11 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:29.421227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 240/481] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:29.426740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:29.427164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:29.427336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:29.446457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:29.447627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:29.536433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 242/482] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:29.571038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd98dc-0658-4b18-bd4d-23ce332f9f11 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:29.690970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd98dc-0658-4b18-bd4d-23ce332f9f11 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:29.736196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:29.737192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:29.737696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:29.738256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:29.757189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:29.762699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:29.763649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:29.764432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:29.767252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 241/483] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:29.773579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:29.871693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:29.885616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2118a97-67b2-4526-9494-3dd42abd55f9 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:29.933060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:29.933793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:29.934092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:29.934358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:29.935076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:29.935375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:29.936736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6ce5-3309-4ee3-a2c4-5f025914994a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:29.945636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 243/484] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:29.951574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:29.981757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.141729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560067b-97c2-4ee4-89d5-11007eaf30e4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.169663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494beaa-03e4-4451-9aeb-799cbb283b07 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.169999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494beaa-03e4-4451-9aeb-799cbb283b07 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.170281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494beaa-03e4-4451-9aeb-799cbb283b07 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.172056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 244/485] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:30.177249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445290b-2f18-4d90-b8fa-a2a9166b32b3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.177535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445290b-2f18-4d90-b8fa-a2a9166b32b3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.177812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445290b-2f18-4d90-b8fa-a2a9166b32b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.195362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.196067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.196289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.196568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.198180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5445290b-2f18-4d90-b8fa-a2a9166b32b3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:30.199559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5445290b-2f18-4d90-b8fa-a2a9166b32b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:30.217431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.220353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:30.248317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:30.249197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:30.250075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:30.250075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:30.250075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:30.250075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:30.254650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 242/486] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:30.255470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.300528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 245/487] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:30.301815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f07e-d343-431a-a1fe-9f71fe52d53e None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.306180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f07e-d343-431a-a1fe-9f71fe52d53e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.306476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f07e-d343-431a-a1fe-9f71fe52d53e None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.307765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 246/488] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:30.314204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.314491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.314772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.338027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.346728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:30.347929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:30.376621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d2970-2b73-4eab-9918-04f4397f561d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.421093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 247/489] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:30.422006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.424776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.425418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.425734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.426080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.426794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:30.427098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.428472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.450196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 243/490] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:30.450966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.461409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.555549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99382b-54c0-41a4-bc3c-8ff59e6fecc2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.589029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79623a9-0de3-49c9-9839-c96a4e3392a8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.599350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.600202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.600622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.601057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.614407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:30.620396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:30.622329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:30.623873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:30.624855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:30.627446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 248/491] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:30.629005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.632765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.634387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.634757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.635110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.649258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:30.655303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:30.656362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:30.657347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:30.661899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 244/492] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:30.662880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.730812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.749550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607fd5-12ec-4c71-96ce-05deebd637c5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.768290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae8c22-7ce1-4e03-ba69-214f1bc886e4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.788473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.789278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.789664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.790066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.798204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.799906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.806606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.807504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.807974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.808418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.809327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:30.809810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.811351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:30.814777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:30.815775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:30.816639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 249/493] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:30.817384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.823645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 245/494] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:30.825519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:30.877039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.930399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b664-62c5-4959-a732-10cce060c732 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.932843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8b1b7-c7e5-43fe-bc6d-7380d39f6333 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:30.967805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.968425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.968712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.969366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.969960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:30.970172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:30.970437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.971179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:30.971682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:30.971845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.972292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:30.973338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:30.974022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:30.975702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:30.982220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:30.985496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 250/495] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:30.986397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.017568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 246/496] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:31.018834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.106558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874de3-721f-46d4-892e-b4264993843b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.134670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b573-b874-4e3e-b0a8-67195906f004 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.143629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.149525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.150303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.150694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.151023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.162624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.164179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.172771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.173413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.173702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.174012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:31.179345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:31.180372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:31.181991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:31.181991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:31.181991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.181991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:31.181991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 251/497] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:31.182342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.189934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.191500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-996c84cc-0645-4443-8a23-d0c3ce0f1371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:31.208678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:31.209727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:31.210813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:31.210813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:31.210813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.210813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:31.214975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 247/498] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:31.216056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184205a2-13a4-478a-9c77-96bd7e479d24 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.216346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184205a2-13a4-478a-9c77-96bd7e479d24 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.216629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184205a2-13a4-478a-9c77-96bd7e479d24 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.217916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 248/499] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:31.221957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184205a2-13a4-478a-9c77-96bd7e479d24 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.243063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.298902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb10bf-a78f-4793-b8f2-3cd040b05431 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.328198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184205a2-13a4-478a-9c77-96bd7e479d24 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.340917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.341937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.342296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.343161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.364889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.365530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.365787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.366127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.366818 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:31.367124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.368492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.384931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 249/500] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:31.384931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.401048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.425635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.427116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d774bea-dacf-4e1d-91be-00c5ed62e0fb None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.477580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.485973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa939c-bc51-4d28-bf03-04ac2db6f8b9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.519632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.520176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.520455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.520733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.521420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:31.521725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.523186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.529898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 252/501] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:31.531142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9a4f-0e87-48f7-92fd-fb92afd4a161 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.531431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9a4f-0e87-48f7-92fd-fb92afd4a161 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.531708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9a4f-0e87-48f7-92fd-fb92afd4a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.539686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 250/502] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:31.540478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.549004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f1df9a4f-0e87-48f7-92fd-fb92afd4a161 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:31.550223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f1df9a4f-0e87-48f7-92fd-fb92afd4a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:31.617012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 253/503] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:31.617537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.619773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a0ae-be2a-4c3d-a0cf-bcc9a95ed8bc None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.620131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a0ae-be2a-4c3d-a0cf-bcc9a95ed8bc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.620433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8a0ae-be2a-4c3d-a0cf-bcc9a95ed8bc None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.625764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 254/504] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:31.625764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8a0ae-be2a-4c3d-a0cf-bcc9a95ed8bc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.645169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59edb-629d-43d6-b20b-38ffe559d4d3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.679969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.680606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.680991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.681171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.689059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.690546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:31.703097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:31.704197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 251/505] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:31.705638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.709397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.730055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8a0ae-be2a-4c3d-a0cf-bcc9a95ed8bc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.764983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.765612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.765921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.766202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.776388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.778453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-52b13fd9-39cf-455c-83f6-547a63b6b706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:31.793445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:31.794382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:31.795299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:31.795299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:31.795299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:31.795299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:31.795299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 255/506] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:31.795882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.796172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.796448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.808075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75c548-033d-40f9-b1f7-62b582e652d9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.813414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:31.814615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:31.841228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:31.841927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:31.842218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:31.842498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] PATH_INFO: `/v3/users/glance` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:31.850354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:31.851814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:31.853581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:31.864893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:31.865808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:31.866762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 252/507] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:31.867851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.884746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 256/508] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:31.885583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:31.947110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:31.980911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f76c3-8235-46de-bc7c-2afcfa67b12f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:31.987379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7bed1-307b-4130-b08d-746cfea8fad3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.013755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.014448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.014734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.015010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.021394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.022035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.022321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.022594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.023393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.023705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.024865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:32.025162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:32.030746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:32.031760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 253/509] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:32.037613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.064646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 257/510] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:32.065426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.093053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.137589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b85a8-9451-4761-b601-dde95a904f4a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.170316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.170923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.171018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d64dcd-781e-48e9-8578-319d45cd0b1a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.171205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.171476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.172171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.172475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.173866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.185346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 254/511] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:32.186159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.191017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.207148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.207787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.208106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.208401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.209264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.209615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.211128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.228391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 258/512] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:32.229188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.286374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1603b-8227-4de1-a940-ce2141be69f4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.303112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.320923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.321559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.321862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.322143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.329030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08099ada-122d-4484-8132-a84fe9f06333 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.332621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.334521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:32.349790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:32.350792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 255/513] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:32.351835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.364790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.365404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.365681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.365989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.366701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.367001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.368368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.376317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 259/514] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:32.377274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.421596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.452258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2132bc-ce67-4f7c-9063-19b24dc7b718 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.478473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57efb3-9d37-4259-a5f2-c1dac96151eb None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.485933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.486549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.487035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.487142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.512553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.513185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.513466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.513742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.520398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.521834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.525192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:32.534174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:32.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:32.535871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 260/515] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:32.536330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.556779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.558782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.635131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.638588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30ade-d0e7-4492-a589-eb6fe30e7037 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.659829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 256/516] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:32.660680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.672412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.673315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.673589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.727935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.741417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.742921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.759656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605cd6-da16-4926-9955-a1b0d4367f12 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.794738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.795343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.795624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.795898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.796584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.796893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.798336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.809368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 257/517] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:32.810113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.844902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.848301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 261/518] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:32.849443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.911479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78065cc2-55ae-424d-a2ea-ddae761d2447 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.939169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:32.945273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.945954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.946244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.946522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.946663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358edb2-1601-4235-a224-eeee6f158533 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:32.957526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.959100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:32.973365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:32.974406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 258/519] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:32.975255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:32.985257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:32.985895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:32.986176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:32.986448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:32.987192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:32.987498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:32.988925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.007133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 262/520] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:33.007918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.068254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.083677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797bf3-ded8-4c35-a6d3-093536f8ad40 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.109518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b45141-6b60-4e6c-ae5f-a5db259fac77 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.117347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.118028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.118313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.118593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.129284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.130780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:33.145690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:33.146637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:33.147464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:33.147464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:33.147464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:33.147464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:33.147685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.147685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.147824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 259/521] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:33.147899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.147899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.148016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.148082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.149439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.171139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 263/522] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:33 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:33.171952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.186683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.264166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d94e-6ec5-4b63-9a83-6d9bd4b2ed24 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.284487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195efa-c5fe-4949-b9ea-9f4762699942 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.301975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.302308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.302938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.303238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.303511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.317029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.318607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.327702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.328336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.329064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.329064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.329798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:33.330743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.331508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:33.334014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:33.334847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:33.336102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:33.336102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:33.336102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:33.336102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:33.336102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 260/523] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:33.336514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.344773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 264/524] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:33.345629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.439186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.457536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d5f3d-99e6-44a3-8512-938d98bfed80 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.460064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82046f-9aab-4998-a506-9064dbdce215 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.495848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.497115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.497539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.497731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.498052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.498310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.498424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.498818 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.499005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:33.499314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.500070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:33.500534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.500677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.502361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.508780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 261/525] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:33.509939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.522675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 265/526] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:33.523997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.578794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.655899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71507f-6a78-4540-aafe-d0dedb61584d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.663273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b052c-6075-4ecd-9835-9758908d03ab None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.695093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.695972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.696664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.696947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.698015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:33.698489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.700096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.701714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.702600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.702940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.703277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.711651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.713179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.714825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 262/527] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:33.718586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:33.731708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:33.732827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:33.733993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:33.736288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.739207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 266/528] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:33.740197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.846852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec9850-a14a-40f5-8ea4-d98e7320daf5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.852564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:33.859272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04760a-852b-42e7-b064-706e8305f2eb None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:33.884236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.884929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.885244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.885907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.893552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.895144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.898473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:33.899384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:33.899875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:33.900357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:33.908014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:33.908882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:33.909696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 263/529] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:33.910197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.915430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:33.915430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:33.931090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:33.931983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:33.932813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:33.935746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 267/530] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:33.936923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:33.987440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.023644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe97c-1d71-4772-b720-9b363e700219 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.053972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3961-c5a9-4fb2-b344-94539972e8f2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.060072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.060952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.061437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.061967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.063128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.063637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.065411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.090636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 264/531] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.093119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.093970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.094664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.094891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.095278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.096053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.096392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.098575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.119400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 268/532] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.120563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.142716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.217458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabb8a7-aa5a-4c5d-ab96-797a2da4cfcb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.249891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af496f-bce7-4ad6-8f15-acab6ab4589b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.250807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.262464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.263071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.263350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.263621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.264710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.265018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.266497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.289528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 265/533] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.289952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.295563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.296471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.296958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.297469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.387589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.389328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.412536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f35858-a8eb-43eb-89bb-8e5bef1a7b7f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.413495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.456161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.457211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.457701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.458340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.459478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b1a7571787a4415fb254a652062e50f3) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.459996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.461813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.468137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 266/534] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/role_assignments?user.id=b1a7571787a4415fb254a652062e50f3&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.469642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.517757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 269/535] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:34.519927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.562396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.640998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443de2ff-f2d6-4b98-b1ac-23da44d3cf39 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.644586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9b64e-0820-46eb-8925-f944fb579556 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.661095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:34.685578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.686773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.687358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.687968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.701209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.703285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:34.722606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:34.723518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:34.724385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:34.724385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:34.724385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:34.724385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:34.730428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 270/536] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:34.733287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:34.759119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.759759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.760053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.760325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.761133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=11a9dc108d68404ba926c566516e7812) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.761519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.763012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.769530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 267/537] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/role_assignments?user.id=11a9dc108d68404ba926c566516e7812&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.860344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37e55-0330-4cda-ad51-28458c2dafe1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:34.900637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:34.901491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:34.902566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:34.903013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:34.904080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:34.904684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:34.906412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:34.925155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 271/538] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:34.930982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:35.064625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde048-ff92-40fa-8684-58119beba367 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:35.082396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:35.125104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:35.125765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.126074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.126353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.139472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:35.140992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:35.152987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:35.153863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:35.155298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 268/539] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:35.159946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:35.191598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:35.266863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351e31-7ee5-4de9-88c3-dab73c37a004 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:35.317382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:35.318049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.318329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.318664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.319430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:35.319736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:35.321232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63c19-db22-46db-a342-0f56473ff92e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:35.336509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 272/540] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:35.342625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:35.456183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b175-fc3c-4412-8647-899b503483bb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:35.479500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a1c96-ffc4-4942-a63d-464e0686d8d1 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.479841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a1c96-ffc4-4942-a63d-464e0686d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.480182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a1c96-ffc4-4942-a63d-464e0686d8d1 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.481794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 273/541] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:35.486951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68084b27-b4c7-45f9-b23e-441649c7d182 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.487240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68084b27-b4c7-45f9-b23e-441649c7d182 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.487524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68084b27-b4c7-45f9-b23e-441649c7d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.500374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:35.500995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.501274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.501548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.502373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c22269afbdf541d4bedbef2cf8503521) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:35.502695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:35.504127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:35.508021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-68084b27-b4c7-45f9-b23e-441649c7d182 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:35.509325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-68084b27-b4c7-45f9-b23e-441649c7d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:35.514826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 269/542] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:35 2022] GET /identity/v3/role_assignments?user.id=c22269afbdf541d4bedbef2cf8503521&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:35.631983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 274/543] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:35.656161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:35.703379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:35.789625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115054-6594-4eb0-8f69-f8da00809f87 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:35.824649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:35.825349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.826819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958fdf-84f1-4e00-aeed-83520754bdc6 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.827106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958fdf-84f1-4e00-aeed-83520754bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.827367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.827472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5958fdf-84f1-4e00-aeed-83520754bdc6 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.827651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.829195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 275/544] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:35.835464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:35.836365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:35.836365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:35.843116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:35.848992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:35.849883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:35.850675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:35.853268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 270/545] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:35.858840 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:35.866696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:35.870481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:35.906191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.004956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 276/546] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:36.027538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.048423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.051864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c339d2-d339-45a2-80ee-5e29109deb23 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.143686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cce11-b01a-485f-922f-2060c7c6824e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.154152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.154791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.155081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.155363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.156217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:36.156400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.157797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:36.179761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 271/547] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:36.183816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.184268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.185112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.185564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.186047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.199124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:36.204840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:36.205695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:36.206510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:36.213688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 277/548] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:36.215599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.232822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.356177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c58c-d0f3-42b3-a822-1b721c3d5b43 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.372455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637f8c-884d-46fe-a141-6a1e573bed5a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.414571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.415182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.415460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.415729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.416434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:36.416735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.418138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:36.436039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 278/549] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:36.442951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.444173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.445058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.445531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.446025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.457928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.459635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:36.481858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:36.483273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:36.484296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:36.484296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:36.484296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:36.484296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:36.489729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 272/550] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:36.495243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.527972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.572325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.628036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30270a46-a6d4-45d5-86fa-27458ed72355 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.678587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae641d-43dd-4c48-a9c8-79cb28fa1806 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.692536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.693776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.694300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.694791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.710041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.711761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:36.732289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:36.733195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:36.734638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:36.734638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:36.734638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:36.734638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:36.742562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 279/551] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:36.745897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.760638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.762128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.762633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.763102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.763991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:36.765240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.766856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:36.793941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 273/552] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:36.798264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:36.939690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8451b-2cf1-4bb8-b0e6-f58fb0b67c46 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.941836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.977028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:36.986311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a9c87-673a-4c9b-bcf0-e4aae02d4f66 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:36.990151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:36.990986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:36.991441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:36.991886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:36.992751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:36.993243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:36.995090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112287cc-cd89-4062-897a-2659e35b2ff4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.013301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 280/553] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:37.014981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177bdb-965c-458e-a8d5-e55cbdd4d745 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.015452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177bdb-965c-458e-a8d5-e55cbdd4d745 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.015965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177bdb-965c-458e-a8d5-e55cbdd4d745 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.017971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 281/554] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:37.018946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177bdb-965c-458e-a8d5-e55cbdd4d745 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.029930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.030579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.031378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.031676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.039874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.041620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-32f1a3d2-3131-4431-8adc-2bc4907f03c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:37.054185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:37.055165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:37.056104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:37.058676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 274/555] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:37.060134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.060677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.061168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.079822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:37.081599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:37.147096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177bdb-965c-458e-a8d5-e55cbdd4d745 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.172703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 275/556] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:37.173895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.197413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.198235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.198746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.199934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.217166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.219316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:37.233897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:37.235121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:37.236610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:37.241601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 282/557] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:37.242574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.251912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.300904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.310587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a194fd-fcaa-40a6-8099-294e6f5c71bb None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.353689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.354396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.354660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.354965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.355752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:37.356055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.357499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.364195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ffc41-9221-47f5-9f03-37907b8d0a26 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.376405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 276/558] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:37.377533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.405072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.405774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.406087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.406363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.417629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-52e7d49d-20fb-468b-a0cf-d72f7b697325 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:37.423604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:37.424713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:37.425910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 283/559] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:37.426580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3cba-f4b2-489b-88d7-ff3f6af46918 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.426580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3cba-f4b2-489b-88d7-ff3f6af46918 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.426873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c3cba-f4b2-489b-88d7-ff3f6af46918 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.430093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 284/560] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:37.430093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c3cba-f4b2-489b-88d7-ff3f6af46918 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.489959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d1a75-a553-4f75-9dfb-e239b4bedd2c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.515208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.529294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.529947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.530354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.530631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.531551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:37.531864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.533406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.537512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c3cba-f4b2-489b-88d7-ff3f6af46918 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.539061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.558747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 277/561] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:37.558747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.580572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.581190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.581472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.581744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.655683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.657121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.665200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45214-0054-4932-b94a-a17223af92f5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.702000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.702621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.702884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.703158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.703871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:37.704176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.705555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b360-1fda-4fb2-9157-90756725064e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.719830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 278/562] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:37.719830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.719830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.719830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.736537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:37.737709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:37.768240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 285/563] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:37.769708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.772781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.785310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:37.818900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 279/564] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:37.820121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.881121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6c918-1f4e-4b16-a9b5-dd64e919674d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.914808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.915439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.915717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.915992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.924479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9e57-4ae9-47c5-b37c-0cfd521a4574 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:37.925670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.927150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:37.942137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:37.943129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:37.944185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:37.944185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:37.944185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:37.944185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:37.944185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 286/565] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:37.944516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:37.964846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:37.965482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.965761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.966058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.976167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:37.977623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-ceef5f29-7e44-4e57-95df-f2862d34fe22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:37.991971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:37.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:37.993881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:37.993881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:37.993881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:37.993881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:37.993881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 280/566] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:37.994326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a303b-2168-4feb-bf90-402c6b5da733 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:37.994655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a303b-2168-4feb-bf90-402c6b5da733 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:37.994930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a303b-2168-4feb-bf90-402c6b5da733 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:37.996720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 281/567] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:37.997401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a303b-2168-4feb-bf90-402c6b5da733 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.005601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.022806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.052008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f5c7a-4638-4cf1-ac7e-fc2ca20ff6e1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.085198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.085849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.086134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.086407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.097538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.098482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a303b-2168-4feb-bf90-402c6b5da733 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.099022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:38.118727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.119622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.120534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:38.120534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:38.120534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:38.120534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:38.120534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 287/568] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.121303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.134846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.135483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.135763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.136053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.146822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:38.152106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.153025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:38.153889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 282/569] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.154574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.222212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c1c6-0be3-45f4-a806-a310d7c796c5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.229003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.235954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.251912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127d7a3-bf2a-4269-9cc9-5e90406d5b72 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.261847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.262494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.262774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.263049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.263777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:38.264080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.265451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333598da-95f4-4393-9aca-843c96365e6a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.276761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 288/570] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:38.278460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.278748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.279024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.289867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.290479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.290757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.291033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.291718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:38.292020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.293403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.296066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:38.297247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:38.304948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 283/571] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:38.305804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.368279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 289/572] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:38.369112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.410712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bee3-da7c-4804-8732-73b745736612 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.446033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.446642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.446925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.447203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.447894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:38.448205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.449605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.460621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 284/573] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:38.461280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.465061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.472436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.479081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ffd3a-4f0b-4735-9955-039c5ac3545b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.512753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.513369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.513651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.513961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.514675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:38.514981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.516342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.524099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 290/574] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:38.530005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.560098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f15e3-4926-4197-a8e6-e6f84c1bdd16 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.595767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.596414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.596700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.596977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.603576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.605042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:38.616193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.617249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 285/575] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.618363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.637553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d931f8-542d-451b-9250-3072ea12b39e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.671083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.671781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.672015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.672302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.680054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.681577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.681872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:38.693430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.694461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 291/576] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.695474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.718884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.724183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd735-bc6c-4b37-bd1b-71f57f6f4867 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.757204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.757866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.758152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.758428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.768363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:38.774176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.775157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:38.775991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 286/577] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.777239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.803717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750389bf-6522-4b8b-a1fd-bb9ecc9d14ec None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.837768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.838645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.838933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.839264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.846752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.848314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:38.866550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.868571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 292/578] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.869634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.879943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51ee5-489f-42fd-84f8-0893b43ab7a5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:38.902664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.913463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:38.914133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:38.914510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:38.914687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:38.925623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:38.927105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:38.941515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:38.942566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:38.943625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:38.943625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:38.943625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:38.943625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:38.943625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 287/579] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:38.944033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:38.949552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:38.972734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8c920-1491-44eb-a267-f2ab35c74e4d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.006354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.006967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.007250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.007519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.008266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.008573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.010024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.021706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 293/580] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.022515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.048397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b078960-91ea-4746-b590-d7e0ec8f2fc0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.083057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.083674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.083959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.084240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.085049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.085388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.086859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.105491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 288/581] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.106330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.124140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200c524-8ec7-441a-9be8-f987ed7f5f36 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.142878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.158316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.158937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.159223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.159615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.160335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.160645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.162042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.167052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.171213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 294/582] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.172131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.213204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d334d-482e-40d2-8be9-657df6efc2d2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.253054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.253674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.253994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.254273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.255032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.255341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.256781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.268724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 289/583] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.269510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.283032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d33cff-9d52-4dce-83e9-fe7f23ced1d6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.318480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.319097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.319380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.319652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.320342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.320649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.322038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.334094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 295/584] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.334816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.363974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.369783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028209-81c5-49b1-9157-09b22e341f21 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.398597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.404081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.404744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.405033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.405367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.419858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=8a4c233f6ff1442b8e08fbe6400ecf0d, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.421312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.434779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 290/585] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:39 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:39.435719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.439537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff150cf-e0e4-4767-8406-201227cff68d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.479704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.480410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.480703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.480977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.495854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.497399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:39.515034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:39.516071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 296/586] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:39.517365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.566058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad425055-bc65-46ac-b16a-0a4c933ed9a9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.606090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.606658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.606944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.607221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.617921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.619367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.622945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.634114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c918-dc38-428b-85ee-b0cc0ecbea7a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:39.636974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:39.638067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:39.639144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:39.639144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:39.639144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:39.639144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:39.639144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 291/587] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:39.639484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.651774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.675249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.675961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.676307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.676648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.677505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=a3dffd8d4deb4cadae3402ffc79ad123) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.677911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.679395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.693862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 297/588] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/role_assignments?user.id=a3dffd8d4deb4cadae3402ffc79ad123&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.693862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.779755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a3d3-1af9-4e16-812e-0ffa3cf81738 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.803919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21514425-3cbf-480f-9cba-1f04428b27ca None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.823438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.824661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.825368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.825653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.837395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.839872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.847542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:39.848226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:39.848556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:39.848825 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:39.849529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:39.849926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:39.851406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:39.853750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:39.855485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:39.856671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:39.862916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 292/589] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:39.863996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.866122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 298/590] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:39.867386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:39.907709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.941955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:39.976894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e266fb-0c2f-461d-8111-bc79f331e6d5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:39.991533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54635c-3f32-4329-9860-45702e7ee96a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.022160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.023184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.023984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.024384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.025428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:40.026096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.028893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.035084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.035701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.035979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.036251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.037042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:40.037348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.038849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.044390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 293/591] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:40.045162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.061829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 299/592] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:40.062826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.174468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.183697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfbfb0-3dff-41e6-966c-4d662c0f44f8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.184975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbce73a-3305-4c6c-abc4-7a4b6efcd5b6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.210270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.226380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.227530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.228178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.228595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.229316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.230162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.230516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.230860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.237246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.238788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.240344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.242472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:40.251021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:40.252056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:40.253146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:40.254936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 294/593] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:40.254936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:40.260022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:40.261201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:40.262418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:40.267899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 300/594] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:40.267899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.377008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e04bc-b187-42d8-b375-0d761e2af467 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.405232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073aba-c652-4dd9-adee-a653e1df2d78 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.415267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.416492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.416873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.418229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.419382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:40.419790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.422111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.439815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 301/595] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:40.441023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.442147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.442789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.443069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.443344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.458132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.459577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.461503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:40.478844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:40.480488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:40.481578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:40.481578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:40.481578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:40.481578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:40.483087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 295/596] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:40.483975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.514522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.561134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf3767-b1e1-4b51-ac39-4ca4f4f74383 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.595829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa473de-e2d5-4c16-a42d-343b2d1912b4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.612827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.613974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.614464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.614962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.616108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:40.616768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.618491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.635746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.636480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.636797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.637081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b1a7571787a4415fb254a652062e50f3/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.641623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 302/597] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:40.642723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.652879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=b1a7571787a4415fb254a652062e50f3, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.654515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.677847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 296/598] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:40 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b1a7571787a4415fb254a652062e50f3/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:40.677847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.771574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.798980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3488f-3545-42c7-baac-b312457d28d0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.815899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:40.866955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.867621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.867901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.868175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.881885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.882991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:40.901637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:40.902677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:40.903611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:40.903611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:40.903611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:40.903611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:40.903611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 297/599] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:40.907864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae8c6-40f4-486d-9e49-8cdc0741d8b6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:40.910912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:40.957091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:40.957737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:40.958047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:40.958328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:40.959036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:40.959341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:40.960761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:40.977937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 303/600] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:40.980587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.026406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525738-0420-4d74-88b3-92656974bdbe None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.058770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:41.064153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.064848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.065561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.065863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.066757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:41.067063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.068426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.080949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 298/601] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:41.086025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.214561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aec2e-77e8-4f5d-8d06-926992bb06f1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.253340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.254071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.254351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.254626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.264639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.266060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:41.279790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:41.280756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:41.281704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 299/602] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:41.286489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5ed94-9d56-4cc3-b333-73cfcf68c406 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.288279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.336030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:41.345607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.346591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.348327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.348818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.365956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.371445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:41.398625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:41.399562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:41.400467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:41.409233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 304/603] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:41.410800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:41.418807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.537580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc6f5-5347-438c-9522-a785f2a72468 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.560175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435629c-e2bf-4ad4-a7ba-6213cd7af7fa None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.575440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.576066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.576348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.576624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.577426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:41.577738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.579241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.595789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 305/604] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:41.603375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.640619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.641242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.641527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.641803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.642581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:41.642891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.644334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.670957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 300/605] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:41.671815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:41.702213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:41.813037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4a2f6-190a-48b6-ba2f-df442b001edd None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.819802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a0a5-05f3-4643-8d7a-ca8ce4a9b970 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:41.835028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:41.849551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.850253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.850535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.850807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/11a9dc108d68404ba926c566516e7812/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.862317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:41.863181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.863781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.864076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c22269afbdf541d4bedbef2cf8503521/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.866206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=11a9dc108d68404ba926c566516e7812, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.867649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f4865-8b0c-429f-9e6d-cacf1f251a53 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.881328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=c22269afbdf541d4bedbef2cf8503521, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:41.882808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f0459-f5bd-4761-b35d-c4f2256fd8fd None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:41.884579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 306/606] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:41 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/11a9dc108d68404ba926c566516e7812/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:41.895321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6839-a316-4b81-823a-3add0cb92594 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.895613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6839-a316-4b81-823a-3add0cb92594 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.895891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6839-a316-4b81-823a-3add0cb92594 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.897597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 307/607] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:41.897989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 301/608] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:41 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c22269afbdf541d4bedbef2cf8503521/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:41.898717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc2b9-e604-408a-9567-e5a90207697e None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.900177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc2b9-e604-408a-9567-e5a90207697e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.900461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fc2b9-e604-408a-9567-e5a90207697e None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.906154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 308/609] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:41.907958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.908276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.908634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.911734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:41.912029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:41.912303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:41.939109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:41.940846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:41.967072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:41.969052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:42.068787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 302/610] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:42.107199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 309/611] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:42.131224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.141136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.176854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:42.254123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054edad0-7231-4e97-808a-bf9bb886f016 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.256252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:42.270283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46048-843b-4547-96bc-1cb158b576cb None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.302142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.302792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.303076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.303351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.325211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:42.332081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:42.333050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:42.333966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:42.334644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 310/612] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:42.340572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.341660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.342565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.342852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.343128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.356628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:42.364422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:42.365723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:42.366731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:42.367473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 303/613] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:42.373168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.507326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:42.524764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946e195-57c5-4e2a-82e0-b35eff2bc351 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.537332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe27f0-194d-4675-8581-b876dbb81a7a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.589755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.590668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.591118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.591549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.592431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:42.592897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:42.595352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:42.602593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.603504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.603504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.603766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.604581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:42.605566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:42.606856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:42.608492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 304/614] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:42.615391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.627878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 311/615] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:42.633629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.646215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:42.749814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71794fcd-990a-4d90-9717-b0241740b95b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.789969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.791262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.791844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.792819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.806204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:42.807984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:42.825304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:42.826892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:42.828001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:42.828001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:42.828001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:42.828001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:42.830277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 305/616] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:42.835839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:42.908974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6d002-88dc-4ba0-8e57-0e10788bbbff None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:42.918520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:42.953508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:42.954370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:42.954650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:42.955306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:42.976403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:42.977865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:42.983433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a62a-97ab-452a-9b81-3cba0784515a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:43.008048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:43.008990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:43.009908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:43.009908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:43.009908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:43.009908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:43.009908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 312/617] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:43.015266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.030803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.052005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.052625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.052902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.053171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.053884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:43.054428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.056187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb579e3b-2c85-463e-86fd-7bc52dd45384 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.073437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 306/618] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:43.073437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5ecf-8f3b-4c2e-bb04-b989726ea711 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.073437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5ecf-8f3b-4c2e-bb04-b989726ea711 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.073437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5ecf-8f3b-4c2e-bb04-b989726ea711 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.073437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 307/619] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:43.077077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb5ecf-8f3b-4c2e-bb04-b989726ea711 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.161695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f12a28-24f1-4cb3-a2a0-97f5166d474a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.196106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.196674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb5ecf-8f3b-4c2e-bb04-b989726ea711 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.196797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.197016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.197253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.197981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:43.198303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.199718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4761696-8f1a-4ccf-a9f5-59adc998309b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.212069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 313/620] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:43.213360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.213555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.213843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.227872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.234802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:43.235999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:43.246393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.247466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.247953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.248567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.257342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.259165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:43.273138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:43.274258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:43.275231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:43.283312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 308/621] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:43.284671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.300999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.309207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 314/622] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:43.314717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.405048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5491-3738-4bc4-a0e2-bd86a3e91c48 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.436482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476abc9b-942b-4cd0-9918-90aea22d17ae None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.444267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.445205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.445776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.446329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.455836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.459023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.472395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.473010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.473290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.473562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.474375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:43.474678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.476084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:43.477780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:43.478740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:43.480273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:43.482989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 309/623] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:43.484086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.494800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 315/624] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:43.500087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.549051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.571731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.604154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b1ab3-f4e0-428e-a910-140e73d7ad89 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.623938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b94-2ad0-4807-aae5-09fd8bdf5943 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.639435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.640118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.640343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.640620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.650974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:43.656253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:43.657185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:43.658055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:43.662181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 310/625] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:43.663456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.669515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.670466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.670962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.671450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.672580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:43.673140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.674808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.698408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 316/626] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:43.699917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.767796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb454f-d40c-452f-abdb-cf34e9a0ccbe None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.771369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.805151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.805836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.806121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.806391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.815715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10c1f-75e0-437f-80a2-7533f85920f8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:43.817984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.819446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.832018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:43.833992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:43.835214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 311/627] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:43.836645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.859153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:43.860124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:43.860630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:43.861108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:43.862614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:43.863421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:43.865672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:43.878583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 317/628] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:43.879699 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:43.956776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df75d-fe56-42f6-a595-37c8848d33c2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.000583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339148-df5e-40e4-a07b-42ce285030b1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.005283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.006206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.006693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.007170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.045149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.048875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.049531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.049807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.050102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.050796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:44.051097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.052452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.065927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 318/629] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:44.066765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.109571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.117135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.122405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.185901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbaab36-a7e2-4b5a-8808-638ae4c244df None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.223951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.224662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.224936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.225519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.236411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 312/630] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:44.236411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.236894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.238665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f040b4d0-1cd8-45e5-9fb7-ac490756fbc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:44.254214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:44.255152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:44.256043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:44.256043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:44.256043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:44.256043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:44.256043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 319/631] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:44.257140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f62e81-b216-4d67-a6b1-5068a504dc8b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.257140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f62e81-b216-4d67-a6b1-5068a504dc8b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.257434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f62e81-b216-4d67-a6b1-5068a504dc8b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.261970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 320/632] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:44.261970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f62e81-b216-4d67-a6b1-5068a504dc8b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.305257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.351342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ccf99-3429-4d2c-955f-a23ad3d4bb41 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.353993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.372042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f62e81-b216-4d67-a6b1-5068a504dc8b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.386459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.387508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.388007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.388490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.396693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.398169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.405075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.405715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.406027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.406303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:44.409211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:44.410277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:44.411155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:44.415189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 313/633] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:44.416209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.416616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.418628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-c9b50a8b-36a9-4d32-9c0d-69c0f73c70e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:44.433664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:44.434650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:44.435518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:44.435518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:44.435518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:44.435518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:44.435518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 321/634] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:44.436078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af5ee9-ffe3-410f-a1e0-959b9bb041d8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.436336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af5ee9-ffe3-410f-a1e0-959b9bb041d8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.436601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af5ee9-ffe3-410f-a1e0-959b9bb041d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.453993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-12af5ee9-ffe3-410f-a1e0-959b9bb041d8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:44.455180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-12af5ee9-ffe3-410f-a1e0-959b9bb041d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:44.521083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b27474-c57a-4ece-9b5e-029a30f44d1b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.522076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 322/635] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:44.523317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dce43f-f526-4fb8-bb15-c721eb056bd5 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.523598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dce43f-f526-4fb8-bb15-c721eb056bd5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.523868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dce43f-f526-4fb8-bb15-c721eb056bd5 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.525083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 323/636] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:44.525757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dce43f-f526-4fb8-bb15-c721eb056bd5 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.550195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.567981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.568593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.568875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.569149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.569863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:44.570168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.571536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.582625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 314/637] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:44.583400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.600240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.646554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dce43f-f526-4fb8-bb15-c721eb056bd5 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.680434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.681050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.681333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.681607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.682392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:44.683231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.684314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0a89c-77be-4c67-9d89-5a6364fce407 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.684651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03361-8638-412b-86be-1b6ad61f04eb None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.696344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 324/638] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:44.697598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.697932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.698237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.717219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:44.718103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.718431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:44.719085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.719367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.719643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.720340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:44.720645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.722021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.733960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 315/639] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:44.734562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.771182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.792628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 325/640] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:44.793470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.837770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e19c78-6a7e-4027-9c86-bed2259ed817 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.859305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.871916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.872556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.872839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.873116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.883092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:44.888469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:44.889681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 316/641] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:44.890784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.907368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c853f9e-8aa6-49d4-912c-01346e04a433 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:44.942154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:44.942796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:44.943076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:44.943346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:44.950824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:44.952260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:44.963679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:44.964770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:37:44.965723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:44.966300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 326/642] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:44.966753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:44.984490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:44.990736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb617-339f-4f44-9019-5bba03633b9a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.023985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.024592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.024875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.025147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/a3dffd8d4deb4cadae3402ffc79ad123/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.039388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=a3dffd8d4deb4cadae3402ffc79ad123, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.040819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.053699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 317/643] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:44 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/a3dffd8d4deb4cadae3402ffc79ad123/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:45.054548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.068427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352713-8d33-46a5-ac23-ee7e0e16b58e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.090311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.114620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.115288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.115634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.115909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.125128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.126433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:45.139341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:45.140327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 327/644] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:45.141372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.188696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2982ed-5a1f-4f58-beaf-dc1c6b73a41f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.231751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.232431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.232727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.232988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.243698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:45.248812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:45.249956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:45.250772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:45.252089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 318/645] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:45.252952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.260417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e0034-a94f-4baa-91a6-f95b8f42cd1c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.263670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.297863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.298388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.298711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.299056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.299829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:45.300139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.301520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.313876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 328/646] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:45.313876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.357017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.367088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c07da-28e8-457f-a310-ecffc3610141 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.407609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.408235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.408516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.408792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.409594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:45.409933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.411382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.423069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4caa8-ca96-4842-9026-a7c938b1acc3 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.429609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 319/647] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:45.430618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.462661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.463616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.464620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.465038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.465989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:45.466453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.468033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.485493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 329/648] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:45.486392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.498356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.538483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741cadf-be92-45f2-aa5a-bce7e27268ee None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.574417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.575040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.575324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.575598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.576315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:45.576618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.578435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.591978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 320/649] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:45.591978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.607314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.610803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397afdb5-3483-449c-9b37-b317b42d1d93 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.647955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.648827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.649102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.649375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.660172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.661846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:45.678624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:45.679664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:45.680767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:45.680767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:45.680767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:45.680767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:45.685139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 330/650] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:45.686010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.709994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16cc4-63c6-4da0-b998-15018261772c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.739054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.749356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.750015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.750303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.750582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.795729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbee4bc-1cc4-4704-9fb7-a0111806e00e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.823419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.825073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.830505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.831128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.831408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.831681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:45.832463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:45.832777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:45.834266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:45.849850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 331/651] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:45.849850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.874235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:45.952161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbaf48-0ca9-4c27-89a1-811f3afb63df None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:45.970045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 321/652] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:45.974953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:45.995964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:45.996798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:45.997084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:45.997358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.012952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.014483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.015759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:46.034044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:46.034981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:46.035860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:46.035860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:46.035860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:46.035860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:46.038127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 332/653] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:46.039251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.121775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.132377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f9738-4fc0-4ff4-a36e-ca2ab8fe2c43 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.160260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756c04e-99f3-44c1-8ddb-971d817a6347 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.173018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.173655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.173975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.174252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.174963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:46.175271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.176689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.190084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 322/654] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:46.190968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.206128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.206795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.207084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.207525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.222293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.224438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:46.243143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:46.244087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:46.244945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:46.244945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:46.244945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:46.244945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:46.244945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 333/655] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:46.250043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.285320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.303381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea382d3-9772-4d4e-a84c-03399d42f4ad None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.339423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.340149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.340501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.340847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.341613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:46.342019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.343456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.356842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 323/656] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:46.357810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.373236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191634-aced-41ce-84f6-5e646a4ca26a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.411252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.412381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.413037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.413435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.422532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.424098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.426637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:46.438905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:46.440101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 334/657] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:46.441295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.476032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfac7b-936f-42f9-b818-71d39271e776 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.514321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.515185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.515640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.516061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.517024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:46.517486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.519091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.531946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 324/658] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:46.533106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.553473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.563172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fcf84-4132-4041-a195-cb22cef3c20d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.606091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.606869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.607158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.607435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] PATH_INFO: `/v3/users/glance` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.616099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.617751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:46.631062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:46.632231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 335/659] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:46.633505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.665670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb4a3-ebb0-4906-bc22-610579109036 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.679925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.705124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.706135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.706628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.707200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.708315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:46.708836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.710501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.730719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 325/660] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:46.732199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.748584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c156477-21c6-4b12-b49a-3ff2735761ea None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.787009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.787665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.787948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.788223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.795623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.797312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.801017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:46.812068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:46.813181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:46.814116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:46.820811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 336/661] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:46.821895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.844332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d966ecf-162e-4b6a-8d2f-2f8557acdcdb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.880777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.881716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.882054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.882427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:46.883323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:46.883733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:46.885358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:46.904570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 326/662] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:46.905582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:46.932072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:46.936309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690ec1e-5c86-4763-bc18-9031e13c7d44 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:46.975144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:46.976091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:46.976630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:46.977116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.013649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65ddd6-3cfe-4af9-a45f-b66501af7975 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.038992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.051869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.052581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.052949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.053324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.054194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:47.055004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.056044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.057076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.059412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.068551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 327/663] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:47.069394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.186295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.189231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c5ba-92dc-463f-8d18-50cdd9413c5e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.192458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 337/664] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:47.193394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.226891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.227980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.228836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.229546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.289225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.299469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d0ea5-a8c2-4c84-bb88-281360531b3c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.309578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.311198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5bb7d-d095-4b77-8b0b-c2330719e582 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.335285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.335989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.336266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.336548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.337331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b1a7571787a4415fb254a652062e50f3) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:47.337642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.339147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11815400-7666-4be2-8ee1-14f69f529ca0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.345429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 338/665] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/role_assignments?user.id=b1a7571787a4415fb254a652062e50f3&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:47.347385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca41b5f-6ca2-4c4a-aa0e-5bd8c55983d3 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.349090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca41b5f-6ca2-4c4a-aa0e-5bd8c55983d3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.349562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca41b5f-6ca2-4c4a-aa0e-5bd8c55983d3 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.356355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 339/666] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:47.356355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca41b5f-6ca2-4c4a-aa0e-5bd8c55983d3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.433985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 328/667] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:47.433985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.433985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.433985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.451689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:47.452838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:47.458210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.512690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca41b5f-6ca2-4c4a-aa0e-5bd8c55983d3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.551300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 329/668] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:47.551300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.564611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.565274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.565650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.566024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.566723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.584003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.586105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:47.602881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:47.604027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:47.605211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:47.605211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:47.605211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:47.605211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:47.605211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 340/669] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:47.605542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.675888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5cdff-3eab-47b0-8154-c9de2ba5f4c8 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.716664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.717394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f241ff2-b3bc-4491-bac1-9a532ebe92f2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.717723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.718181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.718890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.733503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.735483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.736464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.756499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.757226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.757570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.757965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:47.758097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:47.759132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:47.760251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:47.760251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:47.760251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:47.760251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:47.762430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 330/670] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:47.763397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.778016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:47.784665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:47.785882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:47.786927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:47.793670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 341/671] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:47.794638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.844843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:47.894514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c25d-7aac-434f-9a41-ce92be7c17b9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.905181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41503615-18b3-4f4e-b125-f16bc5022265 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:47.941063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.941739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.942089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.942427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.943227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:47.943533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.945325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:47.945459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.946235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:47.946667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:47.947158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:47.948567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:47.949072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:47.950693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:47.963576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 342/673] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:47.963860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 331/673] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:47.964684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:47.966258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.003055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.077713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a9d9-b761-47a4-b9ae-d9952fee5a65 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.083849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb707a3e-82b3-466f-a365-28ad7f4119c2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.116222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.127130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.127785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.128067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.128347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.129715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.130648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.131063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.131502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.132637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.133108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.134499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.136155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.137585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1e28f011-cfaf-43ca-a4d6-e1f436327761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:48.150857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:48.155577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:48.156763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:48.157192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 332/674] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.157268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 343/675] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:48.157337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.157405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eca3-dbd5-42f4-a397-93b25b791c1f None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.157405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eca3-dbd5-42f4-a397-93b25b791c1f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.157405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eca3-dbd5-42f4-a397-93b25b791c1f None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.157405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 344/676] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:48.159949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eca3-dbd5-42f4-a397-93b25b791c1f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.260311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.265061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eebe88-1227-4bd4-856d-f5c30b6479e6 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.267603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eca3-dbd5-42f4-a397-93b25b791c1f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.303533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.303879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.304182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.304466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.304565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.304754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.304852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.305114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.312809 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.314701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.315704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.317127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-7b98a7df-99fd-44be-aa4c-c7c312efad4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:48.326496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:48.327402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:48.328288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 333/677] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:48.328787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.329430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.329430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:48.337618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:48.338542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:48.339968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:48.339968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:48.339968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:48.339968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:48.344284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 345/678] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:48.345102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.347819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:48.349270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:48.392246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.425911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 334/679] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:48.427216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.461418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8822e64-0bce-4eb5-8846-fe7b350a7eee None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.498396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.499011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.499404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.499680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.500549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.500859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.502429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.520223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 346/680] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.521218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.532153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.548542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e07558-f402-4cf5-8647-bec23263c477 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.586343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.586965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.587249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.587524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.588278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.588611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.590102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.607543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 335/681] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.607543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.631478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a46a1e-fb13-475d-8778-0d335b12add7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.647497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.669056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.669682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.670020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.670298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.670990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.671294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.672658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.688210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 347/682] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.688210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.728354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3f79-fe76-40c8-a596-f83056b98d96 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.757639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.765306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.766076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.766433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.766783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.778710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:48.784037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:48.785170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 336/683] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:48.786041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.800822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc15040-6293-4df2-ab20-1cab2fced7a8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.836866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.837481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.837760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.838067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.838907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c22269afbdf541d4bedbef2cf8503521) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.839217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.840638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.846696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 348/684] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/role_assignments?user.id=c22269afbdf541d4bedbef2cf8503521&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.847472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:48.907671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:48.916346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d9e35-1c3f-467b-8c4c-754966e19177 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:48.969980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:48.970625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:48.970964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:48.971254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:48.972531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=11a9dc108d68404ba926c566516e7812) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:48.972847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:48.974364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:48.985952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 337/685] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/role_assignments?user.id=11a9dc108d68404ba926c566516e7812&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:48.985952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.008255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730af418-7632-4b35-9117-118ff12c0751 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.068667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.075900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.078009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.078303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.078583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.125569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5821-37f5-417c-9697-325bb7d7fb30 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.169040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.169689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.170012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.170291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.171081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:49.171397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.172851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.184558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 338/686] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:49.190378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.219739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.223429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.224994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:49.237989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:49.238970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:49.240522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 349/687] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:49.244728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.329783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5cfd2-73ee-4aa6-81d1-ba043fa6494a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.382033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2b9ac-e31f-4bba-96e0-91a51b56d013 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.392827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.393492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.393778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.394561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.409933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.412255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.427789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.428415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.428695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.428967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.429849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:49.430174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:49.431600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:49.433684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:49.435521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:49.435521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:49.435521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:49.435521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:49.435796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.435888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 339/688] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:49.442921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.454631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 350/689] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:49.463916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.538987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.571869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f001f0f-0c7e-4c1f-883b-290a80b68c17 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.586486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.611762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.612377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.612759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.613039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.613731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:49.614065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.615545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad66eb-0026-4755-b56d-74fd3dc7b896 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.629142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 340/690] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:49.630633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a5b3-369c-4336-9d35-b79c04b6a157 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.630918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a5b3-369c-4336-9d35-b79c04b6a157 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.631185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a5b3-369c-4336-9d35-b79c04b6a157 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.632563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 341/691] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:49.637905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a5b3-369c-4336-9d35-b79c04b6a157 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.639061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d6ea9-8f2e-4e9c-b4ce-65bcad46bc2a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.674474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.675098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.675378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.675652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.744651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.746665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.750213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a5b3-369c-4336-9d35-b79c04b6a157 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.787202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.787948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.788290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.788627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.790828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.797663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.799202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:49.805031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e123c479-f627-43d0-b176-6eaa5ea496bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:49.812085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:49.814082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 342/692] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.816414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.818092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.847943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:49.849478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:49.855507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 351/693] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:49.856350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.926959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 343/694] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:49.928901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:49.957165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3575f2-b214-40d3-8675-e38fe8265be8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:49.994933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:49.995551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:49.995828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:49.996101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:49.996895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:49.997198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:49.998650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.020480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 352/695] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:50.020480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.032758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.055236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832774-dc20-4cd9-97d1-908ee0ab65af None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.081761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.095757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.096609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.097045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.097490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.109570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.111313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:50.130716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:50.132072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:50.132963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:50.132963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:50.132963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:50.132963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:50.133222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09300965-e320-45d6-b141-dac6f3cd32fa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.136017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 344/696] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:50.137038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.172636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.173329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.173613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.173916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.174603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.175974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:50.187011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:50.187983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:50.188817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:50.192925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 353/697] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:50.193743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.253366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8103847-bca2-4331-9c1e-0a6aa00e8b83 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.286265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.292644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.293533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.294052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.294534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.313677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df629c5-6fed-4e9b-ba19-879ca45aa678 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.345107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.360876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.361680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.362000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.362275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.363390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:50.363698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.365577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.373479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.375204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.384987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 354/698] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:50.386439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.507045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 345/699] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 12:37:50.507977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.520158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500aa83a-7868-4a04-8f2b-bd0dbcb1ba89 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.559723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.566289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.566923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.567207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.567484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.568433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:50.568743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.570907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.577110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 355/700] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:50.578145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.609665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.634305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77609b3-eb4c-4296-be97-f8d589d1837d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.669774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.670759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.671255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.671736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.680513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.682010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:50.698324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:50.699272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:50.700152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 346/701] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:50.700698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.719440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef490f-f50d-466e-ba86-3fffbe05ce1b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.757413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.758483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.758963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.759432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.770994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.772953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:50.789251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:50.790193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:50.791060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:50.791060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:50.791060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:50.791060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:50.793804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 356/702] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:50.794801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.837713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74960-35ba-4862-a99d-9f74b3881b5f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.851759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.875492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.876454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.876934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.877433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.878390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:50.878900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.880471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.884454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:50.889224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 347/703] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:50.890537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:50.925229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82324e-b4a2-4bd1-ac01-a8b43a647e3a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:50.966102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:50.967177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:50.967670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:50.968149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:50.969365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:50.969909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:50.971582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:50.986416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 357/704] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:50.992974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.020844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969829f-8d8e-4149-b4fd-bbe761d33bea None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.071578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.072249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.072539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.072832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.074251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:51.074679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.076338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.091886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 348/705] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:51.102901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.125516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f8509-1df4-4418-884e-9bc7e95719ba None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.137373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.180683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.181746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.182382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.182858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.183871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=a3dffd8d4deb4cadae3402ffc79ad123) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:51.184423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.186124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3e4ae-d9c9-42a5-abd0-e0551e9d76a8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.194668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 358/706] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:50 2022] GET /identity/v3/role_assignments?user.id=a3dffd8d4deb4cadae3402ffc79ad123&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:51.201053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.216465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dbb52-4f86-4feb-ad34-76818cfb364e None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.216792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dbb52-4f86-4feb-ad34-76818cfb364e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.217039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dbb52-4f86-4feb-ad34-76818cfb364e None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.222688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 359/707] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:37:51.224820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.225111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.225397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.245618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:51.246950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:51.254504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f9081-f401-4ab8-b515-6a290d19e32e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.297343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.298294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.298755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.299185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.308927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.310674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:51.325263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:51.326337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:51.327236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:51.329757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 349/708] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:51.335938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.342573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 360/709] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:51.367481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.481749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e262c6-d750-4593-aa1b-c6147468677e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.496839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9faf5-f7de-4296-91c0-a1a3d953ec33 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.499325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.504013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.525650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.526859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.527143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.527417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.528207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:51.528519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.530599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc28f9-42c1-49ab-8963-27d764e31bc6 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.548431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 350/710] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:51.550428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.550727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d20c0-ed64-4006-8bbd-bb00c9132547 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.551272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d20c0-ed64-4006-8bbd-bb00c9132547 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.551596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.551930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d20c0-ed64-4006-8bbd-bb00c9132547 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.552235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.552671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] PATH_INFO: `/v3/services/network` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.553588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.554896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 351/711] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:51.555087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.556245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d20c0-ed64-4006-8bbd-bb00c9132547 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-476d7db4-a9db-4d35-bd5a-a9e28d473726 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:51.572343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:51.573975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:51.576218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 361/712] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:51.577107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434264f2-971f-4351-876a-4ea710568d03 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.577107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434264f2-971f-4351-876a-4ea710568d03 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.577107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434264f2-971f-4351-876a-4ea710568d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.601427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-434264f2-971f-4351-876a-4ea710568d03 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:51.602663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-434264f2-971f-4351-876a-4ea710568d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:51.685890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 362/713] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:51.687293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434264f2-971f-4351-876a-4ea710568d03 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.687530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d20c0-ed64-4006-8bbd-bb00c9132547 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.736705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.737242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.737524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.737801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.738627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:51.738939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.740396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.753377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 352/714] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:51.753377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:51.852031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.875236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:51.887824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434264f2-971f-4351-876a-4ea710568d03 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.943148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:51.943806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:51.944115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:51.944402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:51.945148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] RBAC: Adding query filter params (name=network) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:51.945484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:51.946968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:51.953487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 363/715] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:51.958321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f1b88-853f-4cd6-9267-1cbdce7cc713 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:51.959465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:52.002525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:52.003870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:52.004351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:52.004908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:52.018775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:52.020664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:52.037330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:52.038331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:52.039254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:52.039254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:52.039254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:52.039254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:52.045926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 353/716] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:52.047594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:52.153300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:52.160744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bc042-d037-4a66-8b4f-1f7a287c5200 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:52.167369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcbe9-9644-428b-86b5-e96d7d024c70 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:52.200343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:52.201356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:52.202315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:52.202784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:52.203674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:52.204172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:52.205843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:52.223884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 354/717] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:52.225443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:52.247122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:52.247756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:52.248038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:52.248314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:52.249004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] RBAC: Adding query filter params (type=network) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:52.249307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:52.251089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b0d64-6750-4b03-9fad-343c4e511785 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:52.265280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 364/718] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:37:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:52.340829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3dab-1d05-4cd0-a6f1-dd89279362f9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:52.381459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:52.382517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:52.383015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:52.383500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:52.384370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:52.386662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88372c-cd59-44f9-9006-71cfc637098b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:52.413997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 355/719] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:37:52 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:37:53.491079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e952-76d3-47a6-82a7-b317bda95fa6 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:53.491574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e952-76d3-47a6-82a7-b317bda95fa6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:53.491574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e952-76d3-47a6-82a7-b317bda95fa6 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:53.493138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 365/720] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:53.499054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbef7-3edb-4ac1-beaf-3662153c3dc3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:53.499467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbef7-3edb-4ac1-beaf-3662153c3dc3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:53.499637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbef7-3edb-4ac1-beaf-3662153c3dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:53.527355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6b9fbef7-3edb-4ac1-beaf-3662153c3dc3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:53.528561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6b9fbef7-3edb-4ac1-beaf-3662153c3dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:53.536847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:53.623502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 356/721] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:53.658394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e952-76d3-47a6-82a7-b317bda95fa6 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:53.689657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:53.720035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ac4e5-7de2-4942-92d9-2b049f79dc09 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:53.720389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ac4e5-7de2-4942-92d9-2b049f79dc09 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:53.720672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ac4e5-7de2-4942-92d9-2b049f79dc09 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:53.722324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 357/722] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:53.727453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:53.727744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:53.728036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:53.747617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:53.748853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:53.830775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e952-76d3-47a6-82a7-b317bda95fa6 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:53.855336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 358/723] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:53.897772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:53.898445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:53.898729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:53.899006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:53.899889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:53.901553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:53.907807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:53.928327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 366/724] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:37:53 2022] POST /identity/v3/services => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:54.027941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:54.063026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ed1d-60f2-48df-b3cd-434266c4f973 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:54.114602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:54.115271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.115859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.115977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.130385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d68e6722-520f-43fe-ad5b-4f372bd029df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:54.136831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:54.138083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:54.139160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:54.141604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 359/725] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:54.147998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.242048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:54.260270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2725130-202a-4fb8-9e86-dbb6548f6597 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:54.298280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10cf9-b783-40da-80ac-a81be8e14cae None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.299090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10cf9-b783-40da-80ac-a81be8e14cae None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.299527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10cf9-b783-40da-80ac-a81be8e14cae None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.301507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 360/726] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:54.304476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:54.305335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.305863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.306347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.307756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:54.308214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:54.310311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.310677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:54.311064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.311539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.322250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 367/727] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:54.327503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.337503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:54.339494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:54.468831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 361/728] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 12:37:54.491164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d4e2-8a5a-46fb-a8a5-16ab7ce4a6da None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:54.498933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.531554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:54.533309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.533868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.534351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.569161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:54.570712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:54.579905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:54.592440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:54.593667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:54.594621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:54.594621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:54.594621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:54.594621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:54.595702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 368/729] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:54.605227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.618837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:54.709122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87eb87-e586-4955-a74d-ea14bc6a777f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:54.755619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36201e-ed01-4495-89b7-dbcb792d3f2e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:54.773985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:54.774866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.775153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.775424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.798280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:54.805551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:54.806541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:54.807539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 362/730] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:37:54.816284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.829017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:54.830025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:54.832758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:54.834752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:54.835508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:54.835818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:54.837212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:54.865689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 369/731] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:54.872228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:54.960879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.017490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cc4ad-f707-48ad-9a8f-e1eebd1dbfb2 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.036511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ffe1-3389-423c-83c5-26e1013e6bc6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.042156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.062271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.062979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.063332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.063675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.064470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:55.064845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.066486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b1600-e05b-42c0-bfd9-1f4391c56bdd None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.076550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 363/732] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:55.081271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.082251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce9c38-3ca4-4823-8f57-1eb847ad6442 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.082615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce9c38-3ca4-4823-8f57-1eb847ad6442 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.082902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.083014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce9c38-3ca4-4823-8f57-1eb847ad6442 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.083480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.083940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.088896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 364/733] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:55.088896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce9c38-3ca4-4823-8f57-1eb847ad6442 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.096933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.098612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-b89f1b44-9d51-4b40-aaed-47a74527a946 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:55.110808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:55.111795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:37:55.112772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:55.115360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 370/734] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:55.117258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.117762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.118252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.139465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:55.140935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:55.204969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce9c38-3ca4-4823-8f57-1eb847ad6442 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.223189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 371/735] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 12:37:55.224638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.251382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.252472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.252879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.253634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.266366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.267824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-4048cf35-2e34-4a1a-abea-475a3c6ae0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:55.285041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:55.286259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:55.287188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:55.287188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:55.287188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:55.287188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:55.291323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 365/736] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:55.291911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d79948-5e6e-40b4-842e-96ee2d1cadb2 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.292896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d79948-5e6e-40b4-842e-96ee2d1cadb2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.293186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d79948-5e6e-40b4-842e-96ee2d1cadb2 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.294718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 366/737] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:55.299237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d79948-5e6e-40b4-842e-96ee2d1cadb2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.300575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.337146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.360334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de52be4-2eba-41e3-ad98-daaf5479ec22 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.396147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.396801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.397056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.397330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.398161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:55.398477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.399902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.412497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d79948-5e6e-40b4-842e-96ee2d1cadb2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.422645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 372/738] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:55.422645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.451784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.452476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.452825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.453165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.453946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.455406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649be8c-114f-4d5c-98ea-e41b43b4c632 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.472481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 367/739] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:37:55 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:37:55.473919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.474403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.474665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.495445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:55.496698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:55.557959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a376-503e-4ce5-bebd-30b51635a5a3 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.570395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.585144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 368/740] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:37:55.586225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.609838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.612032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.613766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.614304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.614770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.615887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:55.616381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.618708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.642509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 373/741] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:55.642909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.718882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cda71-eb33-43d2-803a-23b75e6d9c91 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.764915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.765548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.765861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.766141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.778073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e8956-7c7d-4dd3-9bd1-d402faf83a84 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:55.821772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:55.822494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:55.823322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:55.823612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:55.837033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:37:55.845372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:55.846405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:37:55.847655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:55.849887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 374/742] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:55.850894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:55.864324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:55.866507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:55.885593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.891749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:55.983976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372902d-a43b-46d9-81b4-59f2ee7f7f6d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.003071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 369/743] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:37:56.004635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.026988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.027637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.027923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.028201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.038120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.040032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:56.054971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:56.055942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:37:56.056886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:56.061289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 375/744] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:37:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:56.062160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.136550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873d334-8767-44c5-9439-cf5c7b42fa19 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.155129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.170145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.175229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.175691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.175975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.176254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.176987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:56.177365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.178728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.190051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09eead-c634-4ab8-a275-3589bd76fa3f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.192628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 370/745] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:56.194154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.229724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.230865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.231391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.231868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.244847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.246795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:56.263102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:56.264056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:56.264947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:56.264947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:56.264947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.264947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:56.264947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 376/746] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:56.265232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.303897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153c9c5-b1ca-4d86-807d-843c5a4fab3c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.340824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.341449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.341733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.342035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.343232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:56.343821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.345929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3227ef-e017-42b3-a348-f2ec92e493da None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 371/747] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b8f95-8b87-42c3-8f9a-6083e11fa316 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b8f95-8b87-42c3-8f9a-6083e11fa316 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b8f95-8b87-42c3-8f9a-6083e11fa316 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 372/748] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:56.368394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b8f95-8b87-42c3-8f9a-6083e11fa316 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.376697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf86daa-4f3c-43d4-856e-abd7d8e42d1d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.412007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.412439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.413069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.413347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.413941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.421671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.425591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.427646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:56.441612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:56.442531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:56.443487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:56.443487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:56.443487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.443487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:56.443487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 377/749] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:56.444385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.485437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b8f95-8b87-42c3-8f9a-6083e11fa316 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.526625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.527247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.527531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.527803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.528495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:56.528801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.530211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87657e65-7435-48ce-96f9-3ddb936be4a1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.546757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 373/750] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:56.546757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.546757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.546757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.553560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942e59-d3b6-4dee-adfc-5ea04b199ba7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.563529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:56.565056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:56.592338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.592989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.593267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.593806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.605787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.608080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:56.624978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:56.626427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 378/751] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:56.628122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.648592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 374/752] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:37:56.649328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.678571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.696921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.757372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2be4-6a1f-4f59-94c0-e6b17289aaa3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.760019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaefeb-1d59-42ee-bbbd-eabbda15f52c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.794320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.795037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.795390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.795731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.797021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.797736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.798131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.798509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.799296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:56.799715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.801179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.803064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.804472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.817564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 379/753] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:56.817564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:56.817806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:56.818999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 375/754] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:56.819897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.913220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.926951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87f212-1f33-419e-ba93-808ab02eb5ee None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.928778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:56.929247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1096c-a8ad-45df-95fc-d391d02af776 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:56.971223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.971849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.972135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.972412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.973146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:56.973453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.974874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.975656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:56.976673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:56.977445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:56.977603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:56.978611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:56.980704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:56.994286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 380/755] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:56.994286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:56.996313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:56.997476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:57.004076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 376/756] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:57.005430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.110628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee5b2-25e4-4f64-b146-ffb9c922882c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.122021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e16ae-75c8-4bc9-b315-9603b02a9db5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.148046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.148577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.148862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.149141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.149930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.150364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.151769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.159285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.159923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.160200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.160469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.169693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 377/757] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.169693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.169948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.170355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:37:57.181865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:57.183604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:37:57.185306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:57.188684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 381/758] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:37:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:57.189030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.189503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.215437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.286712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50f90-53cd-4aa9-b659-8e6d120a04c0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.296845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ff017-d09c-42e5-8a65-e84d778af77e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.328243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.328887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.329174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.329452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.331998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.332609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.332886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.333156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.333847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.334151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.335507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.336417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.337873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.345681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 382/759] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.345681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:57.355936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:57.356820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:57.357713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 378/760] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:57.358861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.436149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.437576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.452870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb043864-428f-47ff-b835-b2c60f17c660 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.469806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499a348-fb97-4b4e-a746-497bf3b0fd94 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.490752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.491385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.491664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.491942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.492728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=a3dffd8d4deb4cadae3402ffc79ad123) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.493036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.495027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.500638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 383/761] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/role_assignments?user.id=a3dffd8d4deb4cadae3402ffc79ad123&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.501414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.521403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.522615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.523562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.524022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.525062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.525559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.528567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.561298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 379/762] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.561298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.626535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d0a0-d5dd-4b01-8576-31f69c6d4807 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.664370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.665000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.665281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.665561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.666322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.666636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.668012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.673449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 384/763] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.674733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.692032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.694782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727f5b-3d7f-4730-ba6d-566a13b98086 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.725005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.732962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.734165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.734455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.734897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.735671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.735986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.737416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925970b-5a94-44a7-93ea-90b32a293f3c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.749320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 380/764] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.750724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031f2f6-7ee1-44a5-8dfa-e63d27847374 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.751018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031f2f6-7ee1-44a5-8dfa-e63d27847374 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.751300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031f2f6-7ee1-44a5-8dfa-e63d27847374 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.753932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 381/765] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:57.753932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031f2f6-7ee1-44a5-8dfa-e63d27847374 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.794913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c0657-585b-4b00-b859-38281f3eb013 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.831679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.832323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.832588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.832860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.866132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031f2f6-7ee1-44a5-8dfa-e63d27847374 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:57.900151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:57.900836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:57.901146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:57.901423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:57.902219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] RBAC: Adding query filter params (interface=public, service_id=4f457b3f97a747948e7ca78eedd8b41c, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:57.902532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.903892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.905129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:57.906743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8140a7-c8fa-4d71-a047-c2c44f8dbe4c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:57.916961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 382/766] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/endpoints?service_id=4f457b3f97a747948e7ca78eedd8b41c&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:57.916961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:57.925170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:57.976315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:58.028123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 385/767] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:37:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:37:58.028318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2e4d9-e490-4896-b899-519b3ae28b00 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.029460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.029750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.030074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.048093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:58.049387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:58.069444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.070158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.070442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.070719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.072811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=8a4c233f6ff1442b8e08fbe6400ecf0d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.074824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.097850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 383/768] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:37:57 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:37:58.097850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.149191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 386/769] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:37:58.150361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.212722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:58.240824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fadaa-2331-4650-9c6c-3c00726823fe None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.257839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:58.286912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.287550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.287836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.288226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.288899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.290538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.295760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f392e5a-5552-49ed-9ec2-56b23a0c0a47 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.296445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 384/770] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:58.297056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.336358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.337039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.337315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.337591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.349653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.351177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:37:58.369023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:58.370134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:58.371023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:37:58.371023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:37:58.371023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:37:58.371023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:58.371023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 387/771] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:58.379372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.429036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9cb9b-63ae-4dd8-908d-ab5e2e51734b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.492685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.492897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.493167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.493447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] PATH_INFO: `/v3/services/network` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.494174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.495815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:58.506215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:58.507228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:37:58.508061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:37:58.513863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 385/772] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:58.515329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.534260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:58.544098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6849914-2d1f-478a-b366-0ffacfdaea80 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.571369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:58.594787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.597351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.598557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.599291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.600461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:58.601353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.603327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.623531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 388/773] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:58.632635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.675621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fe92a-5cf8-4659-81f0-a558068aa2ab None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.738814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.739444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.739790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.740073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.740767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] RBAC: Adding query filter params (name=network) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:58.741075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.742501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.751969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 386/774] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:58.757399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.810034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c100d-dd35-4adb-9c41-ccd2bce84119 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:58.907514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:58.908553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:58.909070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:58.909547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:58.924300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:58.925783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:37:58.939763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:37:58.940704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:37:58.941595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 389/775] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:37:58.950657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:58.980487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:59.036577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96aae25-ac1d-40c6-9c7c-f63fb241628f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.044165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:59.096310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.097059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.097374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.097629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.098502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] RBAC: Adding query filter params (type=network) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:59.098948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:59.100328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:59.108297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589e4b4-d425-48d3-93d5-ee289101eb7a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.113872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 387/776] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:59.125006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:59.152901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.153515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.153794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.154091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.154839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:59.155143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:59.156565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:59.179014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 390/777] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:59.184932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:59.294583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd179a-f67e-4313-b08e-c998b098e64b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.341949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:59.356984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.357616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.357920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.358198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.358962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] RBAC: Adding query filter params (interface=public, service_id=77a4e7b874124592992c549b247a6d11, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:59.359281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:59.362341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:59.372535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 388/778] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:37:59 2022] GET /identity/v3/endpoints?service_id=77a4e7b874124592992c549b247a6d11&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:59.377732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:59.392667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23373fd-8459-4dfd-b34c-a962559a0695 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.444688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:37:59.455392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.456059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.456341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.458449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.459422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c26a3ad083a544e996ac6e4dcdc12a37) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:37:59.459745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:59.462517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408b14e-6a58-4d67-852a-5cb2f9654613 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:59.469613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 391/779] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:37:59 2022] GET /identity/v3/role_assignments?user.id=c26a3ad083a544e996ac6e4dcdc12a37&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:37:59.558899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a476-afc9-4e5c-95d4-c639c202d165 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.601469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.602159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.602445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.602741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.603414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:37:59.604967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99add22-1dda-4422-9fd6-c52f029b8dc0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:37:59.611369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 389/780] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:37:59 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:37:59.724029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ef3d-df66-411c-97da-a0ccac469c61 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.724543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ef3d-df66-411c-97da-a0ccac469c61 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.724618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ef3d-df66-411c-97da-a0ccac469c61 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.726458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 392/781] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:37:59.732306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.732592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.732868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:37:59.751820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:37:59.753002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:37:59.823960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 390/782] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:37:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:37:59.847294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56ef3d-df66-411c-97da-a0ccac469c61 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:37:59.957322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56ef3d-df66-411c-97da-a0ccac469c61 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:37:59.993980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:37:59.995241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:37:59.995514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:37:59.995780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.006109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:00.008411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:00.014120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:00.015291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:00.016291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:00.019023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 393/783] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:00.025412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.061130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:00.141399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbeced-75a2-4aa2-9ef5-61ab9f5725f4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:00.179699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:00.180331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.180614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.180890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.181626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:00.182038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:00.183407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:00.196499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 391/784] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:00.202920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.348961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e41b-c365-48b7-a890-35d6cb608b22 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:00.401434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:00.402046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.402331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.402610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.413491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:00.414990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-93348543-06d5-4ad5-8be5-edd57ec31095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:00.431085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:00.432542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:00.434034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:00.434034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:00.434034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:00.434034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:00.434034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 394/785] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:00.438325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.545847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:00.556605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614862e2-e4b4-468e-8d19-0beab2a833e7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:00.606304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:00.607303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.607808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.608279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.609175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:00.609696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:00.611275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:00.622432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0e86-3cdf-45bc-a498-a9d76f0812ed None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.622723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0e86-3cdf-45bc-a498-a9d76f0812ed None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.623027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0e86-3cdf-45bc-a498-a9d76f0812ed None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.624334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 395/786] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:00.625595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 392/787] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:00.631076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.632282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.633172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.634042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.671095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:00.672674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:00.729393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:00.757946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 396/788] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:00.784107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.845325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb050-3246-46ad-9c91-a36f9a86f377 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:00.903693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:00.906470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.909517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.910045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.920895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:00.922976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-f32c9e26-07cd-495a-884b-a717222f3776 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:00.938173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:00.939100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:00.940597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 393/789] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:00.941197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cc83-fa6e-42c2-b341-4db5c125f462 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.941197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cc83-fa6e-42c2-b341-4db5c125f462 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.941576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2cc83-fa6e-42c2-b341-4db5c125f462 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:00.943411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 394/790] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:00.945022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2cc83-fa6e-42c2-b341-4db5c125f462 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:00.947065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef520c-ea50-4c26-be66-ad6141c5ab2b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:00.959789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:00.995970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104637b-e809-4c66-b384-37205249f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:00.996858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104637b-e809-4c66-b384-37205249f541 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:00.997873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104637b-e809-4c66-b384-37205249f541 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:00.998303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104637b-e809-4c66-b384-37205249f541 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.012131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104637b-e809-4c66-b384-37205249f541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-a104637b-e809-4c66-b384-37205249f541 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:01.019475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:01.020575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:01.021420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:01.028626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 397/791] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:01.028626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.028626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.028626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.050375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:01.052793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:01.079822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2cc83-fa6e-42c2-b341-4db5c125f462 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.103008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.127452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.128080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.128364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.128640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.129991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:01.130308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.132051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.144910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 398/792] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:01.146339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.154580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 395/793] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:38:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:01.165847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.238941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.274984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f80c9-ca1a-473b-b8ed-cade94d8ae79 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.290045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d1b00-0402-4af9-8cb0-e7089ca296e2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.311348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.312333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.312765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.313184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.324336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.326080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.328299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.329032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.329411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.329776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.330647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:01.331054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.332507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.346044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 399/794] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:01.346044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:01.347391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:01.348273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 396/795] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:01.349850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.422290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.465089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cacd6-0986-4c4d-8a04-18dde41b024c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.470501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f50e-997a-43e5-94f5-e0b1f4fbd5d3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.491872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.503127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.503958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.504329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.504872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.505638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.507326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.510563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.511372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.511862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.512324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:01.517379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:01.518351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:01.519332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 400/796] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:01.520343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.527815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.529516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:01.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:01.546587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:01.547501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:01.547501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:01.547501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:01.547501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:01.554086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 397/797] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:01.555146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.625352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.630463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64e57-ee74-45bb-9f55-6acb85a5be6e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.663183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c99ba-2831-420f-985b-c9a82d84e1c9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.669780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.670497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.670782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.671061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.671764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:01.672079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.673945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.689766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 401/798] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:01.690856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.704311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.705390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.705932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.706432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.707317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:01.707828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.709469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.715080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 398/799] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:01.716194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.761536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.824277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49efb-51f2-4c17-9ca2-c06e04cf2ee2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.839982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75d5f6-ba4f-422f-9c68-cbca39a2d140 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:01.863348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.864309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.864835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.865327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.866269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:01.866839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.868418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.881583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 402/800] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:01.883119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:01.886554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:01.888360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:01.888858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:01.889360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:01.898682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:01.901728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:01.905058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:01.921043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:01.922031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:01.922946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:01.926158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 399/801] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:01.927106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.015985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd235b0-4883-4f89-9d3c-99f43e75eeee None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.039585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.046389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dba4f-42b2-43cd-8d52-e79e6bc76adb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.058231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.058951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.059234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.059512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.060199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:02.060509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.061909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.071559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 403/802] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:02.072946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.082847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.083489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.083773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.084050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.092364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.094270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-7849fb4a-7c4e-46bd-a2d4-440a9de449a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:02.105394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:02.106430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:02.107388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 400/803] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:38:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:02.107967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4c733-8ec4-4113-b42d-00fa47d71299 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.108127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4c733-8ec4-4113-b42d-00fa47d71299 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.108380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4c733-8ec4-4113-b42d-00fa47d71299 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.111774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 401/804] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:02.111774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4c733-8ec4-4113-b42d-00fa47d71299 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.161956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.180776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a687fa-a7b7-459a-910a-affa099b8891 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.214518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4c733-8ec4-4113-b42d-00fa47d71299 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.218099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.218985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.219272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.219546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.220313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:02.220621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.222073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8a0a7-936b-40bb-9ef8-4e091a837662 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.234124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 404/805] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:02.235304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058553-a1bd-4a7f-9a4d-68e1d4b7a203 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.235589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058553-a1bd-4a7f-9a4d-68e1d4b7a203 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.235862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058553-a1bd-4a7f-9a4d-68e1d4b7a203 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.237327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 405/806] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:02.238042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c058553-a1bd-4a7f-9a4d-68e1d4b7a203 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.252534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.253152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.253435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.253718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.254471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.255843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74218029-19b5-493c-a9ef-320209c9e40a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.277619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 402/807] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:02.278841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.279135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.279417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.314147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:02.315462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:02.324944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.356235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c058553-a1bd-4a7f-9a4d-68e1d4b7a203 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.393015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.393774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.394085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.394355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.395177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:02.395482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.396939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986d7b3-d615-4f9b-b61d-dcbaa5a69add None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.409869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 403/808] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:02.409869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.418261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.422048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 406/809] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:02.423886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.424291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.425228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.445282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:02.446524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:02.520035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 407/810] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:02.520883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.526435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5bf7-9647-49a0-9241-7bb3f1561e72 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.570152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.570999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.571287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.571564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/a3dffd8d4deb4cadae3402ffc79ad123/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.573629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=a3dffd8d4deb4cadae3402ffc79ad123, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.575108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.597853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 404/811] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:02 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/a3dffd8d4deb4cadae3402ffc79ad123/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:02.597853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.597853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.659793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb21fc1-582b-4681-9035-a93578b27a21 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.691571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.701700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.702473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.702978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.703322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.747976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf06d9-728d-4331-9434-0eb40f1385a7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.788022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.789478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:02.796815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.797471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.798234 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.798518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.812724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:02.818416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:02.819432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:38:02.820304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:02.820957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 405/812] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:02.821510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.900744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.902952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 408/813] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:02.904018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:02.933019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:02.948296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda10d9-9940-4be4-aeb4-054053ba7b58 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:02.987187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:02.988333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:02.988820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:02.989315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] PATH_INFO: `/v3/services/network` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:02.990270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:02.992056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:03.002218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:03.003270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 12:38:03.004114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:03.007183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 406/814] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:03.008270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.010914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e535-cf62-4d84-b999-e6e4f2c3a12f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.046924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.047542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.047826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.048105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.049566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.049566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.050588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.059655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 409/815] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.060529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.123659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc6f0-5aed-4fec-86a9-07c9c0599cad None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.153574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.155892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.166156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.167122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.167679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.168157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.170646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c1f1-720b-4c05-9685-b66a1a7fd95d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.180138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.182601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:03.197698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:03.198646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:03.199548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:03.199548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:03.199548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:03.199548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:03.201686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 407/816] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:03.202675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.207973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.208600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.208885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.209162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.210052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] RBAC: Adding query filter params (name=network) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.210362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.211736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.217681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 410/817] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.218988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.314038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a6f58-61ac-472f-ba89-0363fadfeed8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.332746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26b3e1-124e-4e3c-81ef-fcd682b92408 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.350024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.350986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.351507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.352000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.363216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.365149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.370593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.371220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.371503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.371780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.372485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.372792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.374455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:03.382628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:03.383499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:03.384383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:03.384383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:03.384383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:03.384383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:03.386614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 408/818] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:03.387612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.387720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 411/819] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.394824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.414140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.421382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.497219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3323d-65fc-4c4b-b4ec-5900f26f9f5e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.515725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb8c1-2111-4630-aa8f-5ba045c1015a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.532267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.533146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.533631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.534171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.535073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] RBAC: Adding query filter params (type=network) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.535587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.537145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.542334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 409/820] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.543314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.563210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.564350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.564816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.565424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.575059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.577261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:03.591650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:03.592522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:03.593672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:03.600005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 412/821] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:03.600005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.658275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.666842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bdc64-d4b2-4de5-bf3b-33e41250045d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.696961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:03.706135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.706881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.707267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.707668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.708471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.708880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.710533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.723766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 410/822] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.724854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.738903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2643-37a7-41f7-aabe-e994834fccfa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.781533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.782621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.783065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.783450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.784316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.785896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.825891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 413/823] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:03.825891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.895370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226c4d4-255f-4f0c-b7cc-f6ee81bfdc84 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.956920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:03.957621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7eece2-7c60-4243-a092-a4e256bec9c0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:03.958320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:03.958769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:03.959191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:03.960254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:03.960821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:03.962556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:03.982603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 411/824] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:03.983946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:03.993087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.003277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.005977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.008284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.008705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.010114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.022463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.024949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:04.042232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:04.043446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:04.044617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:04.047675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 414/825] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:04.055400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.117586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36395-3a4f-431b-93b0-c835c11f68bb None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.160864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.162075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.162511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.163049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.164391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:04.165211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.168032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.177477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 412/826] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:03 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:04.218035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95ba4-5d8c-45cf-b3d7-478e9b34438e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.258822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.259773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.260207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.260658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.261624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:04.262108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.263713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2998d-df8e-4519-8921-77295b5ca9c8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.286122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 415/827] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:04.292596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.391854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.424412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013bd7c-71db-4646-8790-6728dd402044 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.445897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9b14-0117-41b7-98eb-ae97d9877ae9 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.446176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9b14-0117-41b7-98eb-ae97d9877ae9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.446457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9b14-0117-41b7-98eb-ae97d9877ae9 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.447856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 416/828] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:04.455130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.455558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.455976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.467725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.468695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.468927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.469209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.482711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:04.484185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:04.484308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.485858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:04.502843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:04.503790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:04.504701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:04.504701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:04.504701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:04.504701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:04.504701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 413/829] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:04.510149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.535893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.573119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 417/830] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:04.594922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.625269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b7f9-c71b-4608-b7cd-f17bdf6bf043 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.658327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.663225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.664140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.664619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.665109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.666604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:04.667258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.669008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.683238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 414/831] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:04.689290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.711151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3bd42-d71f-4d9e-876e-aaf491895451 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.748074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.748663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.748952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.749225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.749876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.751292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:04.760881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:04.761907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:04.762725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:04.765937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 418/832] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:04.768527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.805085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c0af-b64b-4759-9915-f89a37dc21e1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.814322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.844057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.844878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.845256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.845537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.852414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.854498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-546d7854-43e9-4461-afbb-4e973e657e81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:04.865763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:04.866880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:04.867986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:04.871561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 415/833] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:04.872844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b2a8-0622-4ce8-b220-bc18b742ed75 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.873116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b2a8-0622-4ce8-b220-bc18b742ed75 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.873376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b2a8-0622-4ce8-b220-bc18b742ed75 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.874803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 416/834] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:04.875334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712b2a8-0622-4ce8-b220-bc18b742ed75 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:04.884123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2e9b0-db6b-4f4b-a4be-127dd8cddbaa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:04.920441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:04.921200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.921571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.921980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.922918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:04.923330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:04.924795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c61d3-fe7e-4b20-8ea0-9a32aa761fd7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:04.929134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:04.930040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 419/835] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:04.932217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:04.932625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:04.932999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:04.951422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:04.952736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:04.994893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712b2a8-0622-4ce8-b220-bc18b742ed75 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.020900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 420/836] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:05.022323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.040117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.041574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.042071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.042614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.043550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:05.044187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.045727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.060071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 417/837] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:05.061132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.071100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:05.148060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70b248-7de3-4bc2-8cfd-ed32d44786de None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.172579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78742-5c4b-48da-a0bb-4b98aaedbf36 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.181082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:05.184103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.184721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.184997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.185269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.185979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:05.186288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.187646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.201849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 421/838] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:05.201849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.225007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.225769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.226156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.226506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.240451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:05.245944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:05.247113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:05.248149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:05.248957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 418/839] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:05.257363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.323278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:05.328483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b21f93-0f90-4c8a-8861-3e91181a05eb None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.367551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.368414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.368845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.369259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c26a3ad083a544e996ac6e4dcdc12a37/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.389098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=c26a3ad083a544e996ac6e4dcdc12a37, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.390821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.403149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d188a6-1478-4933-93c4-def9dd6cf818 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.407989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 422/840] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:05 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c26a3ad083a544e996ac6e4dcdc12a37/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:05.448194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.448834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.449121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.449393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.451367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:05.451367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.451827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.461578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 419/841] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:05.467530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.584483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa63401-66c4-4752-96aa-04e615de1499 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.625494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.626164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.626446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.626713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.637238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.638720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.647628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:05.655874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:05.656884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:05.657764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:05.657764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:05.657764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:05.657764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:05.657764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 423/842] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:05.663122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.692956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:05.775821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56aa60-988e-4212-87b1-2cd141a5e589 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.812356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.812977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.813258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.813533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:05.814268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:05.814569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:05.815927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:05.829282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 420/843] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:05.834852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:05.953422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b65c99-2882-4d5d-be77-57e604f10d8b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:05.993112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:05.994136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:05.994571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:05.995004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.003328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.005674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:06.018623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:06.019872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 12:38:06.021520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:06.023875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 424/844] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:06.033948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.132835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.146676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4fd0-0215-4937-b040-262fa349034b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.181929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.182555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.182810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.183082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.183870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:06.184172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.185579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eac5f5-3c44-4b5e-868d-2aa2ea3b53ce None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.203204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 421/845] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:06.208822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.289804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.339373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5fea-42f1-4720-beb6-0bbbd2b99dd5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.364400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c95b3-eaae-41b5-a70f-464021d59388 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.364703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c95b3-eaae-41b5-a70f-464021d59388 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.364964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c95b3-eaae-41b5-a70f-464021d59388 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.366642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 422/846] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:06.372776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.373001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.373280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.386304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.386926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.387208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.387904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.394989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:06.396301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:06.464954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 423/847] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:06.471063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.472525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.491176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.548484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.582374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.600310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 425/848] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:06.610036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.610252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e8d1-7861-45d9-afee-033161232e81 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.644589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.645159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.645439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.645714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.646386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.647752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:06.657619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:06.659648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:06.661655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 424/849] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:06.666439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.737518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aaef1-55e6-4c4e-9d22-547ca0a35522 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.781047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.782053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.782339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.782673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.795877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.797776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.801229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81925770-e2dd-487c-8c75-8bd20c2d979b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:06.815823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:06.816750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:06.817673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:06.817673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:06.817673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:06.817673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:06.821895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 426/850] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:06.824196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.844269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.851657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.852287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.852571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.852849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.853570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:06.853910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.855842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.862323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 425/851] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:06.867140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:06.894192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:06.940159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b07f5-06f9-4df1-9446-5b1b9d77aede None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.976941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:06.977565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:06.977868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:06.978151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:06.979018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:06.979330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:06.980803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:06.983377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92d7b-bba6-4c5e-89d1-f14f0bf0a27a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:06.994428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 427/852] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:07.000614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:07.027995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.029262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.029677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.030090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.031207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:07.031635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:07.033835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:07.041916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 426/853] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:07.092382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:07.130161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238283-c1b3-4cfa-8f13-e28a5d495e21 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:07.169574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.170545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.170989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.171409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.179373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:07.181167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:07.194892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:07.196037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:07.197212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:07.199490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 428/854] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:07.205581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:07.278577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:07.315356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7356e-06f8-4d67-a552-51542de3300a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:07.352119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.352834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.353181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.354067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.354886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:07.355257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:07.356758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eed311-26d1-4ab1-8275-71ca8c8e59c1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:07.370039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 427/855] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:07.376124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:07.453746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae3148-8c18-46d8-b8e9-a3429bd7f49c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.454008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae3148-8c18-46d8-b8e9-a3429bd7f49c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.454260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae3148-8c18-46d8-b8e9-a3429bd7f49c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.455892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 428/856] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:07.461490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.461490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.461490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.483994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:07.485887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:07.488685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c29b3-a2c3-409a-9e6a-0986e38dd1ea None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:07.525977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:07.532244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.532920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.533197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.533466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.534711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=a3dffd8d4deb4cadae3402ffc79ad123) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:07.535021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:07.536836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17d0c-db37-4d2a-ac07-eaa83f032e95 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:07.547760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 429/857] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:07 2022] GET /identity/v3/role_assignments?user.id=a3dffd8d4deb4cadae3402ffc79ad123&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:07.547760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93aa2-c3b0-4adc-a740-56279cd69656 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.547760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93aa2-c3b0-4adc-a740-56279cd69656 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.547760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93aa2-c3b0-4adc-a740-56279cd69656 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.547760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 430/858] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:07.552081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.552364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.552638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.572407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:07.573099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 429/859] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:07.573661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:07.598859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:07.669959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 431/860] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:07.678686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:07.693596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:07.734593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5204f54-7b9a-403e-bdc4-ad8302d7c17b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:07.777068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.777696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.778033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.778304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.779014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:07.780448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:07.798404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 430/861] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:07 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:38:07.826237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d35159-efd0-4bea-9998-ff6d51405caf None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:07.876207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:07.876931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:07.877216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:07.877499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:07.878337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:07.891008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:07.896417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:07.897409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:07.898304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 432/862] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:07.906143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.018260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511f84-6a2c-4144-8869-3c558438061b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.037189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:08.056889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.057513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.057801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.058102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:08.058833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:08.059137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:08.060581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:08.070290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 431/863] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:08.076939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.182018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81e02-368c-40bf-9e93-7fd91693bfb4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.221074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.221713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.222018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.222293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:08.233245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:08.234771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:08.249859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:08.250777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:08.252296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:08.252296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:08.252296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:08.252296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:08.252296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 433/864] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:08.256740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.371668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664847-a337-4ccf-91d7-17afbaa72111 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.407634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.408261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.408562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.408817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:08.409509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:08.409813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:08.411198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:08.424182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 432/865] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:08.430898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.467689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:08.547123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de172-0cfc-4495-b626-e46a15170cc9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.585705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.586396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.586648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.586917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:08.594981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:08.596419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:08.609789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:08.610814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:08.612416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 434/866] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:38:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:08.616928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.668250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:08.729000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eabc1-16a4-4684-82af-200d06771158 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.765180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.765845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.766140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.766456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:08.767286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:08.767593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:08.769054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:08.788913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 433/867] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:38:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:08.794782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:08.895768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:08.903454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23dd87e-0638-4a36-8a29-6f42d62cc5d9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:08.939168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:08.939710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:08.939988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:08.940259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.014054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.016172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c39f9-eea6-4c41-b4c9-22d483f1aee0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.118246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:09.123714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 435/868] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:08 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:09.133863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.232643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:09.236745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168d1f6-0815-48ef-802e-5de64fc51c29 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.277059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:09.277737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.278028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.278274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.289641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.291214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:09.307787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:09.308734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:09.310139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:09.310139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:09.310139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:09.310139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:09.310139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 434/869] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:09.315426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d17a-70df-4f84-85b9-68d73b002c6b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.316330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.316827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d17a-70df-4f84-85b9-68d73b002c6b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.317305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d17a-70df-4f84-85b9-68d73b002c6b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.319150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 436/870] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:09.326038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.326638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.327116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.345696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:09.347315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:09.425756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0406-757f-4787-aca0-c19d0331758b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.437302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 437/871] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:09.457252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.470020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:09.471421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.471992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.472492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.473547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:09.474082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.476316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.492679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 435/872] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:09.498413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.514019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:09.575625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd36b-11df-4cb3-8c1b-6ba4881e9b83 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.604630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:09.617282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:09.618297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.618835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.619324 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.620202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.621761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45e017-0423-40b1-84f7-5700bb4f5976 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.631146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85995d-604f-4213-8ce0-a052cae625ab None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.643293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 438/873] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:09 2022] POST /identity/v3/services => generated 269 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:38:09.682638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:09.683672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.684191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.684659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.693935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.695790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.702993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31cf0-3030-427d-858f-0d79002e76f9 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.703647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31cf0-3030-427d-858f-0d79002e76f9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.704121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31cf0-3030-427d-858f-0d79002e76f9 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.706330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 439/874] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:09.711624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:09.712655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:09.713549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 436/875] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:09.718942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.718942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.718942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.720317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.739738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:09.741290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:09.797074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:09.830360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 440/876] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:09.835535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ec3b-c5aa-4065-8733-7247250d2632 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.847026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.872892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:09.873779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:09.874263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:09.874545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:09.875307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:09.875646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:09.877058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:09.894960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 437/877] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:09.899307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:09.960459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce427c56-e5ce-448b-bc70-6e439c0a5938 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:09.977514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.001374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.002329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.002706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.003069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.003955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.006144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.012978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4d4cf-bc1b-4651-a9ea-98c55404be42 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:10.016215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:10.017304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.018227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 441/878] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:10.023891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.045345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.055413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.056834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.057350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.058013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.059115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c26a3ad083a544e996ac6e4dcdc12a37) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.059611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.061379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9e1e3-b124-4652-bf02-77a6c5cf9f9a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.068601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 438/879] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:09 2022] GET /identity/v3/role_assignments?user.id=c26a3ad083a544e996ac6e4dcdc12a37&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:10.070187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35419f-1e0f-4857-b4d9-bb93f3d4ca40 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.070666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35419f-1e0f-4857-b4d9-bb93f3d4ca40 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.071254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35419f-1e0f-4857-b4d9-bb93f3d4ca40 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.072889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 439/880] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:10.079673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.080288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.080784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.103758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:10.105527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:10.142987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc43d-867a-45ce-90b2-08add84afea6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.184601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.185263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.185541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.185811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.186646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.187022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.188503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.198218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 440/881] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:10.199278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 442/882] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:10.200180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.218782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.268835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.312880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e841a9-6a90-4406-a163-01e0cc2b9430 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.338206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.348883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1775-6585-4354-bc36-af19e74260f8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.352705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.354311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.354852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.355369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.356259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.356782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.358458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.364161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 441/883] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:10.388273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.388948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.389234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.389510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.390203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.391642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:10.401924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:10.402985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:10.404276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 443/884] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:10.409560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.523641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fe069-e26d-47d2-8e77-d4ce67f5ea8d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.559741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.560499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.560890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.561236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.562026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.562401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.563886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.569221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 442/885] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:10.575505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.692706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c8338-7c24-4e0d-8df8-9100d44608ba None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.701269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.730273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.731250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.731743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.732231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.733225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.733779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.735804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.742378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 444/886] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:10.748832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:10.802704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:10.857382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec62a2-d42d-4f52-9b8e-cfc1af2ae36a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:10.893315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:10.893971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:10.894255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:10.894571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:10.895341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] RBAC: Adding query filter params (interface=public, service_id=d0fbbd6f99b44940a3704b8124c0662b, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:10.895652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:10.897013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd711fba-b5c3-4b7b-88f6-015c65956213 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:10.903452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 443/887] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/endpoints?service_id=d0fbbd6f99b44940a3704b8124c0662b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:10.908673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:11.027761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96ad6-d264-4ab6-938c-2502262c1eee None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:11.071809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:11.072554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:11.072845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:11.073122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:11.073848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:11.075226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f41f7-921a-4ef5-968c-402e9b8cfbff None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:11.081306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 445/888] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:10 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:38:11.850067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28df-fbe4-4dac-abcd-ae0dfcbcd593 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:11.850623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28df-fbe4-4dac-abcd-ae0dfcbcd593 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:11.850623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28df-fbe4-4dac-abcd-ae0dfcbcd593 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:11.852187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 444/889] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:11.858187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:11.858610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:11.858804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:11.877627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:11.878840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:11.907596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:11.955665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 446/890] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:11.978856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a28df-fbe4-4dac-abcd-ae0dfcbcd593 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.086239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a28df-fbe4-4dac-abcd-ae0dfcbcd593 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.100389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:12.123370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.123654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.125851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.125851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.125851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.129857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2efa49b6-4081-4807-be02-e5d52121004c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:12.136161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:12.137083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:12.138108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 445/891] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 29 12:38:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:12.146910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.257263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695342-faf9-4910-8cdc-9a9ab3d54698 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.295200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.295860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.296146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.296439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.297151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:12.297455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.298150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2921-63e4-497d-8701-78669b3baf12 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.298653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2921-63e4-497d-8701-78669b3baf12 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.298883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.299324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2921-63e4-497d-8701-78669b3baf12 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.301127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 446/892] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:12.307548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 447/893] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:12.308982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425521-a32d-4514-b008-bb260e735dab None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.309333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425521-a32d-4514-b008-bb260e735dab None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.309679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425521-a32d-4514-b008-bb260e735dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.315688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.330651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-19425521-a32d-4514-b008-bb260e735dab None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:12.332163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-19425521-a32d-4514-b008-bb260e735dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:12.352055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:12.406821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 447/894] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:12.424975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c273f-54c1-456a-9c6c-1d45f9014c79 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.425270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c273f-54c1-456a-9c6c-1d45f9014c79 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.425538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c273f-54c1-456a-9c6c-1d45f9014c79 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.427284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 448/895] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:12.432761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.433044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.433323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.449778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4138e17-16d3-4b70-92f5-fd607d1fedc9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.451432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:12.452599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:12.487369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.488401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.488874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.489392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.490101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:12.490410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.491776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.501956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 448/896] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:12.502901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.527731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 449/897] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:12.528456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.565419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:12.627026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:12.636425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc0a0f-e356-4555-b047-e9935c05938f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.636927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bf14-1a1e-4d2a-ab68-718c628eabf4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.674960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.675880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.676258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.676535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.677207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.678026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.678483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.678907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.679818 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] RBAC: Adding query filter params (interface=public, service_id=c92ddc1dd1f5420ca4fc4e0079d4abf2, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:12.680284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.682400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.687636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:12.688739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 450/898] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/endpoints?service_id=c92ddc1dd1f5420ca4fc4e0079d4abf2&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:12.693611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:12.693704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:12.694695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:12.695586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:12.699166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 449/899] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:12.699838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.796910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:12.799901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f1355-2a76-4dbf-ad50-64635d5c7c69 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.815888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938447-5502-4fef-a13f-3ca2ae03ae14 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:12.835516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.836133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.836411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.836681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.837387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.838886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.854134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:12.854756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:12.855153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:12.855541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 451/900] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:12 2022] POST /identity/v3/services => generated 241 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:12.856276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:12.866157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:12.866959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:12.868373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:12.878058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 450/901] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:38:12.997469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0f1f-395b-4a29-9a46-adb09762524a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.040110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.041316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.041602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.041913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.042750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:13.043313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.045698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc352464-a54f-47f2-845b-5be970f10414 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.057635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 452/902] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:13.063680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.088410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.182145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b639b-0792-433a-b95f-ab36286901d0 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.182392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b639b-0792-433a-b95f-ab36286901d0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.182635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b639b-0792-433a-b95f-ab36286901d0 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.184003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 453/903] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:13.189161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.189523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.189917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.192484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46136ea8-1e04-47b4-9aad-922877b188b9 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.211785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:13.213594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:13.228713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.236630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.237871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.238386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.238852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.252685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.254275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:13.270375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:13.271343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:13.272213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:13.272213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:13.272213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:13.272213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:13.276973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 451/904] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:13.278503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.297153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 454/905] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:13.320414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.366058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.417537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4993b3-d5ce-48c0-a1c9-be806fe4ac71 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.442011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b7df6-2bfe-4a14-a211-717aea1a23f1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.465363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.466040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.466320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.466592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.467576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:13.467889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.469787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.484593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 452/906] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:13.490044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.491264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.491934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.492217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.492494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.493141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.495344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:13.507189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:13.508273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 12:38:13.509114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:13.513679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 455/907] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:13.515401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.535269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.609195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a153d-a95f-4f86-8a49-13f6ae0c950c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.641415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.646206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498fcc-bc99-46c6-867f-a979e4907734 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.655368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.656784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.657671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.658226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.669069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.671329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.683129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.683834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.684191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.684538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.685302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:13.685673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.687146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:13.689091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:13.690100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:13.691072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:13.698299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 456/908] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:13.698448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 453/909] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:13.701160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.704450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.786819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.812151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558da6-2114-4304-b624-c99ccc71d547 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.820485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cd8f8-9da8-4319-9429-915ac5e9443a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.847012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.847700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.847903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.848187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.849024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:13.849286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.850766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.860328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:13.861213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:13.861715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:13.862223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:13.863170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:13.863682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:13.865390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:13.873158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 457/910] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:13.875131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.877365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 454/911] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:13.882425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:13.941297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:13.983204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf031ab-a23d-45a0-81b7-fed699154342 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:13.997684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9af24-a270-414e-b269-ac2d282f8b81 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:14.022940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.023710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.024104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.024509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.036101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.036924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.037371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.037961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.038913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.040582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.055354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:14.069869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 455/912] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:38:14.120754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.122588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.237513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 458/913] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 12:38:14.243592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:14.358837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d6190-53e1-4248-be6f-258180412754 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:14.361310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:14.396319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.396965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.397250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.397523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.408930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.410411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:14.425281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:14.426200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:14.427101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:14.427101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:14.427101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:14.427101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:14.427101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 456/914] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:14.434216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:14.474197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:14.549908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856805da-53e3-4f0f-a859-d2d80d17b8fc None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:14.585341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.585999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.586284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.586559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.587254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:14.588100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.589479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.601756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 459/915] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:14.607867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:14.716670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99272ca8-09f2-47fa-8359-412fe94104f1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:14.751868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.752510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.752791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.753066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.760318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.761789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:14.773841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:14.775317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:14.776207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 457/916] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:14.784246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:14.893037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf7cc-2c2f-42c2-8ebd-da223b586920 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:14.910210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:14.929488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:14.930145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.930426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.930701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.931491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:14.931799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:14.933230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9b864-f82d-492c-8a4d-6c4a5dacb63f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:14.946079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 460/917] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:14.952866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:14.976581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:14.989519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6345-56dc-4b21-a769-8f17ac08ec0c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.989802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6345-56dc-4b21-a769-8f17ac08ec0c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.990108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6345-56dc-4b21-a769-8f17ac08ec0c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:14.991590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 461/918] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:14.996856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17341-ee80-42ce-8649-a431812d8a18 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:14.997146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17341-ee80-42ce-8649-a431812d8a18 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:14.997425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17341-ee80-42ce-8649-a431812d8a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.015400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f3d17341-ee80-42ce-8649-a431812d8a18 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:15.016587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3d17341-ee80-42ce-8649-a431812d8a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:15.077416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ae56-d94b-43a8-b14c-7cff590a705d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.097252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 462/919] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:15.098703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6df4c-d615-4740-af19-832d55efe110 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.098993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6df4c-d615-4740-af19-832d55efe110 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.099268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6df4c-d615-4740-af19-832d55efe110 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.101090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 463/920] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:15.107366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.107657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.107933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.116664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.117618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.118132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.118648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.119632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] RBAC: Adding query filter params (role.id=3cb6a7557a1e4eb99f23da0d8fea5536, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c26a3ad083a544e996ac6e4dcdc12a37) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.120289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.121852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.126715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:15.128094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:15.130000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 458/921] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:14 2022] GET /identity/v3/role_assignments?user.id=c26a3ad083a544e996ac6e4dcdc12a37&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=3cb6a7557a1e4eb99f23da0d8fea5536 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:15.136000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.211761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 464/922] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:15.235322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.235702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.252726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.261760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafe500-8744-4b8c-bdda-883b6e1f9598 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.319744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.320403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.320689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.320964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.321599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.323763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:15.335388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:15.336478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:15.343852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 459/923] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:15.344705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.380607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4752052-52fc-48a0-bba8-49c410448590 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.418428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.419129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.419413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.419694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.420588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.421886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:15.431610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:15.432672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:15.433517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 465/924] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:15.438988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.487346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0191829-d1f4-4f7e-acda-6cb08fa2c443 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.527821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.540196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.541522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.542298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.542785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.544376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.544901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.547443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.552420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c740ac-23e2-47bb-bf92-7214dd951d70 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.558837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 460/925] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:15.559976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.571494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.594245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.594686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.594973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.595251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.595945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.596256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.597639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.603922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 466/926] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:15.609070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.672914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58f52-6898-4776-8081-44e00d3c6120 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.709661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.710665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.711153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.711611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.712564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.712827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6b7e-672b-4199-a41a-15d28cc800e9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.713318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.715009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.725535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 461/927] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:15.726644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.758978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.758978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.758978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.758978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.759891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.759983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.761321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.772959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 467/928] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:15.778205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.792466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.817064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:15.852513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a13bf-4b96-4a2f-a57b-08773a09ee74 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.890386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.891490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.892073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.892557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.893543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] RBAC: Adding query filter params (interface=public, service_id=4b1461411ac7411fa845013bad5567fd, region_id=RegionOne) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:15.894092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.895760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.909342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 462/929] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/endpoints?service_id=4b1461411ac7411fa845013bad5567fd&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:15.910323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c35b-1ce2-477c-9de8-f2985f263279 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:15.911097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:15.948633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:15.949479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:15.950543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:15.950543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:15.951153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:15.952585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1917bb6-af96-403e-8ed1-9197b423b3a2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:15.981909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 468/930] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:16.033901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92287e0c-fc66-4099-b12d-a359591b5f45 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:16.083726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:16.091697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:16.092330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.092612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.092885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.093883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:16.095383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d18d1b-b901-464c-ac52-1f9e22fdc40e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:16.102866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 463/931] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:15 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:38:16.266875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ec9f-572a-4fd9-af6b-9531b0a8b820 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.267623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ec9f-572a-4fd9-af6b-9531b0a8b820 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.268166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ec9f-572a-4fd9-af6b-9531b0a8b820 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.270238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 469/932] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:16.276010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.276305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.276573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.296244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:16.297760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:16.377761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 464/933] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:16.400539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6ec9f-572a-4fd9-af6b-9531b0a8b820 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:16.466100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:16.512815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6ec9f-572a-4fd9-af6b-9531b0a8b820 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:16.558099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:16.558740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.559080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.559361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.560142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:16.562038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:16.565579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING keystone.server.flask.common [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 12:38:16.585331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.sql.core [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 12:38:16.585331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 12:38:16.585331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [parameters: {'id': '73310aca5a1b4b67a0a02a70a140ec81', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 12:38:16.585331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91819) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73310aca5a1b4b67a0a02a70a140ec81.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73310aca5a1b4b67a0a02a70a140ec81. Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:38:16.602561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application result.read() Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 29 12:38:16.603973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application yield resource Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise self.value Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application session.commit() Jul 29 12:38:16.604950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self.session.flush() Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application compat.raise_( Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 12:38:16.606420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 12:38:16.608591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application result.read() Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:38:16.610598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [parameters: {'id': '73310aca5a1b4b67a0a02a70a140ec81', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:16.612941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:16.623147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73310aca5a1b4b67a0a02a70a140ec81. Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:16.625300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 470/934] 10.222.0.66 () {60 vars in 1140 bytes} [Fri Jul 29 12:38:16 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:16.626591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:16.717256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b4d82-34ef-4618-b839-7ba96734ebbd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:16.746426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:16.755474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:16.756174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.756453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.756725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.764054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:16.765581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0699c611-060d-420d-9e46-dfd17f519e7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:16.782510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:16.784579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:16.786718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:16.789849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 465/935] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:16.790141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:16.895897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47b91-3738-475b-8308-13d5e7b9a391 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:16.914350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:16.932521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:16.933150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:16.933438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:16.933715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:16.934476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:16.935145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:16.936837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46cc1a-25eb-46b4-b2db-17895824b646 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:16.962006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 471/936] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:17.231949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d728c92-ca8e-453a-8a21-9d9c0027f7f5 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:17.232435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d728c92-ca8e-453a-8a21-9d9c0027f7f5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:17.232435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d728c92-ca8e-453a-8a21-9d9c0027f7f5 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:17.234041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 466/937] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:17.239309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:17.239603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:17.239884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:17.258576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:17.260686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:17.331157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 472/938] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:17.354184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d728c92-ca8e-453a-8a21-9d9c0027f7f5 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:17.463816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d728c92-ca8e-453a-8a21-9d9c0027f7f5 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:17.500121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:17.501006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:17.501445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:17.501891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:17.513578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:17.518945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:17.520057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:17.521100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:17.523464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 467/939] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:17.531800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:17.633461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06a9d0-5044-428a-9565-780116834da1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:17.644992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:17.671024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:17.671707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:17.671992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:17.672269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:17.672984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:17.673290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:17.674686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:17.683861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 473/940] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:17.689510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:17.739236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:17.800879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde89fa-f2df-4430-a8e1-a322840cb071 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:17.837334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:17.838003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:17.838284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:17.838555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:17.849288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:17.850757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:17.866092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:17.866983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:17.868026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:17.868026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:17.868026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:17.868026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:17.868026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 468/941] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:17.872956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:17.975900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb90d48-80bf-4770-8eee-93cfa64af65f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.012248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.013199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.013698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.014186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.015076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:18.015560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.017106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020f7f-d51b-4891-9348-494bd56f97c4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.030044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 474/942] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:18.039085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.125180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfb6-b4e9-49c7-9ea1-826b3141e178 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.125691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfb6-b4e9-49c7-9ea1-826b3141e178 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.126204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfb6-b4e9-49c7-9ea1-826b3141e178 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.128111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 475/943] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:18.133850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.133850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.133850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.146044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.149806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbe020-449c-4172-9f38-b49363053fff None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.155135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:18.156581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:18.186424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.190600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.191778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.192218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.193066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.201190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.202919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:18.217806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:18.219332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:18.220492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:18.222502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 469/944] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:18.232276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.238251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 476/945] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:18.261649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.347760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60613dc1-a7dc-4c65-9ea1-45192ca01841 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.382083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fecd-2c5b-4ca5-9e64-f6326e13d354 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.384291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.384905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.385186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.385461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.386314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:18.386623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.388040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff19a-8c01-409f-805f-a43a98d612bf None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.406029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 470/946] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:18.407215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183059-69a2-467b-bfe6-b7556e455ddf None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.407503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183059-69a2-467b-bfe6-b7556e455ddf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.407779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7183059-69a2-467b-bfe6-b7556e455ddf None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.409131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 471/947] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:18.413629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7183059-69a2-467b-bfe6-b7556e455ddf None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.425320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.426347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.426837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.427318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] PATH_INFO: `/v3/services/compute` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.428359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.430110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.431402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-79c424c5-2b69-4cea-89af-eb9ddc466f4a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:18.441863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:18.442849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:18.443681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:18.451174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 477/948] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:18.451174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.451476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.451746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.470457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:18.471641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:18.490368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.527963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7183059-69a2-467b-bfe6-b7556e455ddf None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.543109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 478/949] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:18.543947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.570979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.572247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.572524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.572797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.585193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.587511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:18.608983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:18.610211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:18.611113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:18.611113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:18.611113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:18.611113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:18.611750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 472/950] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:18.612573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.651293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c33f00-4c3b-4d82-a6d0-8ffb49bd52df None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.682209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.688547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.689166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.689454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.689735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.690458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] RBAC: Adding query filter params (name=compute) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:18.690772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.692148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.698300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 479/951] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:18.703009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.720878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ddbac-5b48-4daa-a2d7-4162de3baf9c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.723408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.762413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.763062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.763346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.763626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.764258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.765649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:18.774837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:18.775834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:18.776678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 473/952] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:18.777485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.821122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9441-467c-46ca-a579-b7518d7b060f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.859161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.860207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.860207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.860380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.861015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:18.861326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.862719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.881091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 480/953] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:18.881091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.892144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4384df2-62de-406b-ad20-fbdcc2d341f0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:18.927356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:18.933111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:18.933727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:18.934037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:18.934310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:18.934988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:18.935288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:18.936646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:18.941461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 474/954] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:18.942257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:18.996223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:19.014868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff498-abeb-4185-aad6-9b2f94c844d7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.069868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.070576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.071145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.071431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.072201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:19.072507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.074763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.082765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 481/955] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:19.084665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a3a14-38dc-40da-a46c-155d4f850367 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.088780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.135480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.135688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.135844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.136104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.143422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.145232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0858795d-d4c6-47bd-9a5a-b5438b652ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:19.158314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:19.159527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.160778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 475/956] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:19.161464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871550ed-282a-47b3-bfba-4ab24706487d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.161464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871550ed-282a-47b3-bfba-4ab24706487d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.161683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871550ed-282a-47b3-bfba-4ab24706487d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.163241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 476/957] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:19.168158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871550ed-282a-47b3-bfba-4ab24706487d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.212787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9696e-5e81-42c8-b764-82ea1e1174a5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.229447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:19.250389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.251039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.251301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.251576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.252253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:19.252556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.253951 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d28b8-294e-4bc4-beb3-853c4ea5b38e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.259592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 482/958] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:19.263264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.264167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.264671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.278175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871550ed-282a-47b3-bfba-4ab24706487d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.278986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:19.285254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:19.286485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:19.314400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.315018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.315296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.315754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.316402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:19.316614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.318190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.330573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 477/959] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:19.331704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.366249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 483/960] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:19.367077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.440011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce54b-87b8-4063-8ab2-e5482f3edcb7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.458950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:19.479842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.480397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.480681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.480781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d0847-1578-4513-9517-bbec34f6e219 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.480965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.481674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.483083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.503629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 478/961] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:19.504504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.529680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:19.535399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.536685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.537169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.537704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c26a3ad083a544e996ac6e4dcdc12a37/roles/3cb6a7557a1e4eb99f23da0d8fea5536` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.540898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=c26a3ad083a544e996ac6e4dcdc12a37, role_id=3cb6a7557a1e4eb99f23da0d8fea5536)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.542764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.565924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 484/962] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:19 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/c26a3ad083a544e996ac6e4dcdc12a37/roles/3cb6a7557a1e4eb99f23da0d8fea5536 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:19.662316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beab48-5f7f-4ad2-bcea-dc4318a65290 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.708953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.709597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.709931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.710211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.710907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.712511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.715160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING keystone.server.flask.common [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 12:38:19.730332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.sql.core [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 12:38:19.730332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 12:38:19.730332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [parameters: {'user_id': '091e49e121a24cc3aafc4b9cf0131516', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 12:38:19.730332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91820) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result.read() Jul 29 12:38:19.737981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:38:19.738979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application yield resource Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application next(self.gen) Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise self.value Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application session.commit() Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 12:38:19.740068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.session.flush() Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application compat.raise_( Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 12:38:19.740972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 12:38:19.742689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application conn.query(q) Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application result.read() Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 12:38:19.745279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [parameters: {'user_id': '091e49e121a24cc3aafc4b9cf0131516', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:19.747458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 29 12:38:19.750212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:19.751733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 479/963] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 29 12:38:19 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:19.752221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.863422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbebb3c-e705-41e4-bbff-de4658d551ce None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:19.900657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:19.901618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:19.902068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:19.902489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:19.910157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:19.911777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e1cd308c-fe17-4661-87a5-ba5e48c9d087 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:19.927322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:19.928495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 12:38:19.929751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:19.932093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 485/964] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:19.938386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:19.976571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:20.067055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b1849-7df2-455e-bf9f-149bee50f912 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:20.108177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:20.109177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:20.109662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:20.110158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:20.111157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:20.111667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:20.113531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df88bc-0500-4d61-9eea-dfc52e5f05ef None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:20.153666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 480/965] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 29 12:38:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:20.969515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038caaa-474b-413c-a4e6-fe1b2eb54d67 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:20.970097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038caaa-474b-413c-a4e6-fe1b2eb54d67 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:20.970097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038caaa-474b-413c-a4e6-fe1b2eb54d67 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:20.971579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 486/966] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:20.976531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764172-fe83-45e2-97e6-24e8c2d717cb None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:20.976907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764172-fe83-45e2-97e6-24e8c2d717cb None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:20.977096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764172-fe83-45e2-97e6-24e8c2d717cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:20.995939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a5764172-fe83-45e2-97e6-24e8c2d717cb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:20.997425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a5764172-fe83-45e2-97e6-24e8c2d717cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:21.065837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 481/967] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:21.089889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038caaa-474b-413c-a4e6-fe1b2eb54d67 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:21.154706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:21.205001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038caaa-474b-413c-a4e6-fe1b2eb54d67 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:21.238462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:21.238998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.239275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.239602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.240303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:21.241652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f92f7-a5a0-49e3-a248-44c284d0f5af None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:21.265858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 487/968] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:21 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:21.646001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a2d-b8b4-4c6e-8a8f-7e06c7d1425a None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.646332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a2d-b8b4-4c6e-8a8f-7e06c7d1425a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.646566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a2d-b8b4-4c6e-8a8f-7e06c7d1425a None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.648063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 482/969] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:21.653221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.653512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.653788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.672323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:21.673509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:21.699569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d546802-9785-4ff0-af5b-30f2efec6891 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.699950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d546802-9785-4ff0-af5b-30f2efec6891 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.700115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d546802-9785-4ff0-af5b-30f2efec6891 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.701465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 483/970] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:21.706816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.707302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.707591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.728476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:21.729681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:21.748360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:21.753634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 488/971] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:21.776536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:21.803024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 484/972] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:21.816971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:21.829671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:21.888320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b582171-2e0e-48e2-8308-bd444f3e7057 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:21.926452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:21.927331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.927615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.927888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.928518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:21.929906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:21.934823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f6ef0-8bde-4695-85e9-cfede9c81ec0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:21.938968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:21.939998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:21.941168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 489/973] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:21.945899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:21.977461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:21.978196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:21.978473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:21.978747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:21.990836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:21.997012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:21.998235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:21.999100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 485/974] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:22.007185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.023869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.071116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bc4ed-d38c-4e41-adfd-9b471dee8d88 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.073513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.111412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6912f-3989-40cb-b8fb-06c3f6dd27e8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.113285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.115073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.115599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.116087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.116996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.117573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.119752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.131839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 490/975] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:22.132853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.150602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.151495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.152016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.152297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.153063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.153372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.154775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.163191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 486/976] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:22.168644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.248584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e702c-e0bc-472e-bee4-a7fc6c9ef83e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.249665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.278912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789009-11c9-4ad2-95e6-d5bdc16ff687 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.285454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.286113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.286397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.286676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.287353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.287657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.289055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.294197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 491/977] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:22.299779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.319635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.320278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.320562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.320842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.332261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.333795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.333970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:22.350476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:22.351430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:22.352328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:22.352328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:22.352328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:22.352328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:22.352328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 487/978] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:22.357858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.417409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4cd76-d35a-4865-9f66-fed2e8b2d6a9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.456977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.457715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.458272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.458453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.459176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] RBAC: Adding query filter params (interface=admin, service_id=4b1461411ac7411fa845013bad5567fd, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.459491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.461094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.466654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba294-0fd7-4769-ba60-700ac7251324 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.471495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 492/979] 10.222.0.66 () {56 vars in 1249 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/endpoints?service_id=4b1461411ac7411fa845013bad5567fd&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:22.472435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.496593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.506486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.507415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.507703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.507978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.508663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.509502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.510925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.527197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 488/980] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:22.528264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.575359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.583250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9ca7-e6c8-4cfb-86ab-2abb2c3fdf17 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.618474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.619136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.619427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.619705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.620366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.621740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.628370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 493/981] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:38:22.649267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7fa59-a4d6-440b-be11-7032e0d42071 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.687641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.688306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.688586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.688863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.698041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.699621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:22.712599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:22.713608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:22.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 489/982] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:22.719964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.831146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182c56d-d638-47ee-be98-0a31d12c12e1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:22.864294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:22.864917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:22.865199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:22.865474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:22.866301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:22.866610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:22.868029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:22.886878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 494/983] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:22.892747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:22.941959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:22.999283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1b92c-6c80-4ace-a74a-2fbe4ccefb37 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.034341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.034961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.035244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.035517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.105283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.106757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9442e9-9db5-4527-9219-70c6fdafa63f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.158739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.218378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 490/984] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:22 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:23.220505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.239223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.278896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dfc44-8677-4ef1-b0ea-655668f2d2be None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.279209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dfc44-8677-4ef1-b0ea-655668f2d2be None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.279444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dfc44-8677-4ef1-b0ea-655668f2d2be None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.280964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 491/985] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:23.285430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.285717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.286025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.306121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:23.307293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:23.327043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6a4f-2934-4e6b-a20f-718ca6e6db0c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.366078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.366635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.366914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.367185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.375368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 492/986] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:23.380188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.381711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.397386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:23.397796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:23.398692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:23.399711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:23.399711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:23.399711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:23.399711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:23.400261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 495/987] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:23.404341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.471828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.484686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.509732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96487330-ba3f-4cf5-8fff-ace77c83cb63 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.510350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289d943-c44a-403e-83ae-40e64ee0674e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.545358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.546051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.546203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.546377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.546603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.546706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.546936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.547211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.547305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.547905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:23.548212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.548584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.549566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:23.559185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:23.560219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:23.561063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:23.565285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 493/988] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:23.566143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 496/989] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:23.567021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.567182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.670950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697e8a-a648-4a58-95ea-f89b552dfa82 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.674341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d7fb-62e9-49ba-996d-72a7a0aee3b5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.698967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.709404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.710031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.710306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.710648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.710762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.710927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.711200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.711294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.712404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:23.712706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.714096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.723445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.723941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 494/990] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:23.724781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.725070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.729194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:23.737096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:23.738023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:23.739111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:23.743781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 497/991] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:23.744745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.835302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5fee9-927f-4569-b122-a4c6924f1a3a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.858857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf7dd9-9adf-4843-9e97-fa87759e9ed1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:23.873443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.874213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.874592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.874977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.875746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:23.876168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.877622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.888241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 495/992] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:23.889719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.893682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:23.894342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:23.894623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:23.894895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:23.895647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:23.895956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:23.897373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:23.916309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 498/993] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:23.918015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:23.960446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.974017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:23.996849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af011e6-6a63-476b-90ef-7100483e8f75 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:24.032047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3e93b-d5c0-47b7-ac25-04ae90706682 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:24.032666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:24.033408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.033780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.034173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.035017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] RBAC: Adding query filter params (interface=public, service_id=a415f9c287f54dffb04a1f039a760b4a, region_id=RegionOne) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:24.035420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:24.036868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:24.047481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 496/994] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/endpoints?service_id=a415f9c287f54dffb04a1f039a760b4a&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:24.048467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:24.069000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:24.069621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.069928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.070211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.071983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] RBAC: Adding query filter params (role.id=3cb6a7557a1e4eb99f23da0d8fea5536, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=c26a3ad083a544e996ac6e4dcdc12a37) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:24.072584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:24.074231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387668d0-1071-4377-aa16-4fd6a86e47e0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:24.093861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 499/995] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:23 2022] GET /identity/v3/role_assignments?user.id=c26a3ad083a544e996ac6e4dcdc12a37&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=3cb6a7557a1e4eb99f23da0d8fea5536 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:24.166672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f644-b01b-4aef-8b5e-83be932de1d2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:24.199100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:24.205734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:24.206450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.206753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.207053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.207735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:24.209147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0960a2-6975-4eaa-8a2f-81aa9aed812b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:24.226195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 497/996] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:24 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:38:24.654463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff014c6-3b58-4ca5-b0cf-5f92b58a3923 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.655007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff014c6-3b58-4ca5-b0cf-5f92b58a3923 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.655007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff014c6-3b58-4ca5-b0cf-5f92b58a3923 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.656587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 500/997] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:24.661726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.662050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.662334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.681628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:24.682878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:24.760851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 498/998] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:24.809022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff014c6-3b58-4ca5-b0cf-5f92b58a3923 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:24.932846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff014c6-3b58-4ca5-b0cf-5f92b58a3923 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:24.947268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:24.970303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:24.971273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:24.971764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:24.972244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:24.973106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:24.974727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:24.984321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:24.985344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 12:38:24.986231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:24.989114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 501/999] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 29 12:38:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:24.997746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:25.125435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baba1dd-2782-4645-b4f1-c8584535ec07 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:25.161082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:25.170710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:25.171334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:25.171605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:25.171877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:25.172547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:25.172851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:25.174241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:25.184165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 499/1000] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:25.185318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:25.290725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e3fcc-314f-4451-a30d-fb4a7020a66b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:25.324849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:25.325472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:25.325758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:25.326066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:25.326752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:25.327059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:25.328425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42339119-47eb-4837-8c14-007e4b59d151 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:25.333533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 502/1001] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 29 12:38:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:25.338983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:25.444316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10c109-c2d8-4620-af37-91c2e78ad7e7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:25.478873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:25.479492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:25.479780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:25.480059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:25.480775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:25.482175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce12cc-bcf4-4a04-9760-bccb47dfc074 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:25.504560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 500/1002] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:26.138911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694690-f114-4722-88e6-4cc241455b26 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.139389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694690-f114-4722-88e6-4cc241455b26 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.139476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694690-f114-4722-88e6-4cc241455b26 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.140985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 503/1003] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:26.145902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2e350-75d4-4706-a1bc-4228ca9ac146 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.146303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2e350-75d4-4706-a1bc-4228ca9ac146 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.146516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2e350-75d4-4706-a1bc-4228ca9ac146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.164988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3da2e350-75d4-4706-a1bc-4228ca9ac146 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:26.166194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3da2e350-75d4-4706-a1bc-4228ca9ac146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:26.212402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.242115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 501/1004] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:26.264282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4694690-f114-4722-88e6-4cc241455b26 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.268772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485db6-daab-42ad-935e-3539206c00a5 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.269069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485db6-daab-42ad-935e-3539206c00a5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.269351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f485db6-daab-42ad-935e-3539206c00a5 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.273858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 502/1005] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:26.276391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.276682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.276960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.296303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:26.297576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:26.308788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.367605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 503/1006] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:26.375510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4694690-f114-4722-88e6-4cc241455b26 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.390175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.412117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.412790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.413079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.413346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] PATH_INFO: `/v3/services/image` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.414002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.415441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:26.424816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:26.425858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:26.426726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 504/1007] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:26.432520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.499146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ac673-a078-495e-a64e-a6718aeff8f4 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.508032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.538916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.539561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.539843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.540119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.540746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.542133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.545444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c980e0-19e5-4139-9c61-95a43464a932 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.548560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:26.551230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:26.552214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:26.553060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 504/1008] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:26.558909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.593944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.594678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.594966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.595243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.596051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] RBAC: Adding query filter params (name=image) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:26.596359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.597724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.603353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 505/1009] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:26.608957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.667639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c7f7-d321-4b08-b81a-450c10f35119 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.703891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.704930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.705219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.705645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.706372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:26.706684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.708127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.713010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 505/1010] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:26.718861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.721966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6066d8c-c4c4-48ac-be78-dfd33fca7304 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.754719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.760479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.761418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.762004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.762286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.762972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] RBAC: Adding query filter params (type=image) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:26.763274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.764626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.777850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 506/1011] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:26.838669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8952-42b0-4518-ae99-74db1b39cc71 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:26.875125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:26.875815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:26.876102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:26.876379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:26.877150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:26.877464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:26.878884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22708f46-538e-423f-9899-e8ea25a6cb70 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:26.889902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 506/1012] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 29 12:38:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:26.891104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:26.925806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:26.998662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf2a01-692e-4014-a335-5925b0587dcd None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:27.032903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:27.033520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:27.033812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:27.034100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:27.034805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:27.036160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc1e87-a622-40fe-b37a-5ca6b40ed07b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:27.062062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 507/1013] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:27.597059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e239a2-d34c-4ecf-8e50-252eb0617a46 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:27.597585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e239a2-d34c-4ecf-8e50-252eb0617a46 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:27.597585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e239a2-d34c-4ecf-8e50-252eb0617a46 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:27.599146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 507/1014] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:27.605797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:27.606282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:27.606515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:27.625532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:27.626732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:27.699620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 508/1015] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:27.721730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e239a2-d34c-4ecf-8e50-252eb0617a46 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:27.821812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:27.826925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e239a2-d34c-4ecf-8e50-252eb0617a46 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:27.864950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:27.865519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:27.865802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:27.866118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:27.867267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:27.869256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63fc74-8bab-4e98-ada4-bcc0d5150ff3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:27.871426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 508/1016] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:27.877872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:27.892640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:27.980454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b771a57-b0b2-476d-8566-9b2d3af56694 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:28.013763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:28.014396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:28.014674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:28.014946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:28.015626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:28.017032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8030c-cf67-4b5c-b2c6-723d779dfd7f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:28.044528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 509/1017] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:27 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:28.819821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d9d6-e184-4aba-ad7b-84d8157c4289 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:28.820370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d9d6-e184-4aba-ad7b-84d8157c4289 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:28.820370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d9d6-e184-4aba-ad7b-84d8157c4289 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:28.822095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 509/1018] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:28.827401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9042ce-d146-4eeb-8af2-ce8f7733b147 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:28.827845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9042ce-d146-4eeb-8af2-ce8f7733b147 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:28.828016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9042ce-d146-4eeb-8af2-ce8f7733b147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:28.849140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2c9042ce-d146-4eeb-8af2-ce8f7733b147 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:28.850375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2c9042ce-d146-4eeb-8af2-ce8f7733b147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:28.913446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:28.929492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 510/1019] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:28 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:28.954079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1d9d6-e184-4aba-ad7b-84d8157c4289 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:29.059613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1d9d6-e184-4aba-ad7b-84d8157c4289 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:29.094309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:29.094855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.095136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.095408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.096105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:29.097465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cabe9-ac49-4e93-987d-6c168a80035d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:29.129953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 510/1020] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:28 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:29.176807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecab44-b7c3-4056-a7d8-3b1c8c966b7a None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.177154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecab44-b7c3-4056-a7d8-3b1c8c966b7a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.177357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecab44-b7c3-4056-a7d8-3b1c8c966b7a None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.179105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 511/1021] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:29.185550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.185934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.186140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.205748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:29.206983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:29.230840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:29.304228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 511/1022] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:29.327573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecab44-b7c3-4056-a7d8-3b1c8c966b7a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:29.433808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecab44-b7c3-4056-a7d8-3b1c8c966b7a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:29.468710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:29.469358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.469646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.469956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.480346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:29.485242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:29.486206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:29.487041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 512/1023] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:29.494398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:29.596969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41385b8-859f-41cd-84f9-396cc33afc1f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:29.634233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:29.634859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.635137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.635409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.636119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:29.636421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:29.637803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:29.650314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 512/1024] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:29.652241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:29.711503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:29.756465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672d04-c7aa-45e1-abea-a19110396b26 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:29.792446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:29.793094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.793380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.793658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.804072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:29.805709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:29.820275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:29.821700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:29.822654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:29.822654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:29.822654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:29.822654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:29.826148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 513/1025] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:29.828058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:29.846098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:29.934844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5ec0-6f6c-4b7a-8681-a1a1cc981149 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:29.967629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:29.968422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:29.968720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:29.968995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:29.969762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:29.970088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:29.971440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056846c-9589-4250-9e3e-89ba7be010f0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:29.987900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 513/1026] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:29.989541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.091764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d7a50-2a2e-4766-8cd2-867554754e53 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.098565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e94-2c0d-4a83-a164-bc5a2d297525 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.098860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e94-2c0d-4a83-a164-bc5a2d297525 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.099140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b03e94-2c0d-4a83-a164-bc5a2d297525 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.100495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 514/1027] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:30.101532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.106455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.106741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.107013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.127796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:30.129116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:30.136021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.136662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.136940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.137214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] PATH_INFO: `/v3/users/nova` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.144810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.146282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:30.157246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:30.158227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:30.159278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 514/1028] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:38:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:30.163975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.208061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 515/1029] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:30.232740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.235710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.275894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f50ee-99a3-438b-b8b0-e3263d16018c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.312757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.313376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.313657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.313973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.314766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:30.315081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.316913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.335365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 515/1030] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:38:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:30.341560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.342827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b82c-55d7-41b2-97b4-a38bb25e95ab None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.374145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.384027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.384667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.384945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.385214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.386404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.387822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.393811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 516/1031] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:30.395086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.455937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8e735-816a-4619-921e-970059c289f8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.467799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.494345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.494974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.495302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.495583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.499567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34ae3-297a-47c2-8e0d-a172f01f7e36 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.534209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.534819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.535119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.535368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.536035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.537447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.561184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 517/1032] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:38:30 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:30.569210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.570776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.603597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.690048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 516/1033] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:30.691349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.799639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12826a85-ebce-435f-9e0e-948dc9b419f8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.825985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:30.835819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:30.836547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:30.836762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:30.837037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:30.848102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:30.849549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:30.864265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:30.865218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:30.866155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:30.866155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:30.866155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:30.866155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:30.866155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 518/1034] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:30.872296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:30.972834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f37a-fb89-4dde-bd1d-db07b08bd698 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:30.998766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:31.009698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.010347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.010627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.010901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.011591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.011896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.013261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0329-c067-4f7b-abf5-cabb12b53beb None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.029192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 517/1035] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:31.030818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.141023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145ef-d604-4cb3-8cb3-25dc090f513f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.161057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4ae8-d708-41fa-bdc9-cc462d9e07e9 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.161346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4ae8-d708-41fa-bdc9-cc462d9e07e9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.161624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4ae8-d708-41fa-bdc9-cc462d9e07e9 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.163183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 518/1036] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:31.167814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.168110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.168388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.176560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.177227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.177536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.177810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.185544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.187083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.187685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:31.188935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:31.199522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:31.200599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:31.201539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 519/1037] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:31.207082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.251057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:31.265080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 519/1038] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:31.287214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.311095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33b69-57b7-48d2-a189-a8eac084cedc None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.346286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.346907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.347191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.347457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.348206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.348513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.349967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.362328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 520/1039] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:31.368379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.395836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79595-378d-4af5-b03c-5bfb50d35646 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.413338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:31.433185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.433836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.434130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.434405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] PATH_INFO: `/v3/services/image` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.435119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.436595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:31.445453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:31.447025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:31.447890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 520/1040] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:31.453279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.470967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:31.474123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190ad6-a52b-4ca9-83c1-ee663f54d001 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.512668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.513279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.513559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.513843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.514638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] RBAC: Adding query filter params (role.id=3cb6a7557a1e4eb99f23da0d8fea5536, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.514956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.516405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.522040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 521/1041] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=3cb6a7557a1e4eb99f23da0d8fea5536 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:31.561922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f22a0-4562-4825-a60a-4809a40e468f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.599429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.600069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.600351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.600623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.601320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] RBAC: Adding query filter params (name=image) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.601625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.603100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.608878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 521/1042] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:31.614768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.729608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207f68-a6fa-43c5-bd8a-8286008f0187 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.767374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.767995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.768273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.768541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.769214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] RBAC: Adding query filter params (type=image) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.769513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.770910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.780476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 522/1043] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:31.782069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.832201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:31.887393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaee294-b0ed-4e3f-b103-01252d1c6bfa None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:31.922347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:31.922953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:31.923231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:31.923501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:31.924257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] RBAC: Adding query filter params (interface=public, service_id=d981b469ee09449e839732ca2305978c, region_id=RegionOne) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:31.924563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:31.925933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313133-55e1-498e-aabe-a6d58aabbe53 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:31.936020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 522/1044] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/endpoints?service_id=d981b469ee09449e839732ca2305978c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:31.937265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:31.968212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:32.043248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e011b-55e9-4540-a5a9-16de4e420c11 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:32.078843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:32.079466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:32.079743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:32.080012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:32.080656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:32.082029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381ae72-68ac-42f8-aa59-17ef8f4c2f78 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:32.093853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 523/1045] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:38:31 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:38:32.644478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13b2e-131e-4ead-ab61-7b1cdddf627c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:32.644996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13b2e-131e-4ead-ab61-7b1cdddf627c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:32.645103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb13b2e-131e-4ead-ab61-7b1cdddf627c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:32.646585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 523/1046] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:32.652063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:32.652366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:32.652644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:32.670565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:32.671744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:32.738945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 524/1047] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:32.761883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb13b2e-131e-4ead-ab61-7b1cdddf627c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:32.864091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb13b2e-131e-4ead-ab61-7b1cdddf627c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:32.872849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:32.898626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:32.899813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:32.900096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:32.900369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:32.910916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:32.916327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:32.917220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:32.918464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:32.922158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 524/1048] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:32.923539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:32.948200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.026103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b23ae-964d-45fa-bfb8-a8f875d1b4f6 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.059455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.060078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.060361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.060635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.061347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:33.061652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.063036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.071974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 525/1049] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:33.077466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.187697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31859ab9-ff1c-4833-ac5c-2a9584beb3ac None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.224767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.225438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.225845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.226164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] PATH_INFO: `/v3/users/9afaf46b4a13420fa14c898baaa26712` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.241022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] RBAC: Authorizing `identity:get_user(user_id=9afaf46b4a13420fa14c898baaa26712)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.242500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24077b25-269b-48ce-86df-c35bcc84fdf0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.249475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 525/1050] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/users/9afaf46b4a13420fa14c898baaa26712 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:33.250644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.337050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.353636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574665-95b3-446b-bea8-5ee03057c841 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.387778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.388393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.388670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.388941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.455312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.456685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.513568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02eaa-68bd-498e-a02d-1f25dacf13c1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.513976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02eaa-68bd-498e-a02d-1f25dacf13c1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.514258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02eaa-68bd-498e-a02d-1f25dacf13c1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.515635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 526/1051] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:33.520396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.520685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.520960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.544380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:33.546245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:33.547204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.564140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 526/1052] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:33.569628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.572111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.624569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 527/1053] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:33.646903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.674579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426182f-e56d-4ca3-9fc4-f77c58968428 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.715295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.716463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.716749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.717033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.718999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.721312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.727895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 527/1054] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:33.729599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.751006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1b9b1-9e08-4cfe-9f0c-496bc051f5cc None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.787973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.788618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.789860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.790153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.790491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.801293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:33.806284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:33.807205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:33.808062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:33.812344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 528/1055] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:33.813193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:33.832366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:33.844229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112599d0-c67f-46ce-a7b0-34c000643142 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.880034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.880784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.881052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.881443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.882248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=8e873b2933c64d74b86445d73388ad92, user.id=9afaf46b4a13420fa14c898baaa26712) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:33.882554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.884556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.890468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 528/1056] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/role_assignments?user.id=9afaf46b4a13420fa14c898baaa26712&scope.project.id=8e873b2933c64d74b86445d73388ad92&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:33.918961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7f77-1e60-42ab-9b2c-e63bd34149e3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:33.957655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:33.958602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:33.958887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:33.959162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:33.959967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:33.960273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:33.962261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a357504-8f86-404b-a867-b4ea82ccc79f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:33.976794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 529/1057] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:33.978637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.089418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17add645-3dfe-4ace-82be-12788330bef3 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.106253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.130862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.131513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.131795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.132071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.135114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2c7c-51ab-4f59-8997-1c41fc01e8c5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.135409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2c7c-51ab-4f59-8997-1c41fc01e8c5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.135688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2c7c-51ab-4f59-8997-1c41fc01e8c5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.137166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 530/1058] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:34.141738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.142058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.142340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.144049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.145566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.160436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:34.161060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:34.162184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:34.163076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:34.163076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:34.163076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:34.163076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:34.163413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:34.163490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 529/1059] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:34.169377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.236471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 531/1060] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:34.258365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.268872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.274459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7330e7-a790-43b8-a4be-16070991af8d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.313033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.313653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.313977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.314251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.314939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:34.315241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.316591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.330026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 530/1061] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:34.338788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.369852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.376488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f057c6d-8980-407c-8070-9424c0381910 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.413876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.414557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.414842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.415118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] PATH_INFO: `/v3/services/image` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.415745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.417107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:34.426320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:34.427479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:34.428711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 532/1062] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:34.433452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.456109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7708f-8360-4236-abf9-e445d1bd8aec None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.494282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.494899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.495211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.495518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.503861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.505344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.517292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:34.526570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:34.527597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:34.528542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 531/1063] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:34.533271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.554064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3ce5-2e68-431a-a15d-754c1b296a47 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.591557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.592177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.592457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.592728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.593673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] RBAC: Adding query filter params (name=image) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:34.594007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.595359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.600511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 533/1064] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:34.606630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.628733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.642605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c8da-9566-4a57-9d3e-3812bc599c97 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.678277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.678989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.679247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.679460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.680309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:34.680617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.682126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.699471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 532/1065] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:34.705044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.717209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1d0dc-8505-409f-8454-553ff900ab26 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.757486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.758062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.758369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.758642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.759308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] RBAC: Adding query filter params (type=image) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:34.760027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.761638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.764729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.768820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 534/1066] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:34.774799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.816359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba37c8-de05-4317-8a40-c0127726838e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.852886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.853905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.854905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.855374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.866036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.867476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.874845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:34.883780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:34.884843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:34.885960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:34.885960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:34.885960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:34.885960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:34.886180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a80bb5-11b5-46c2-93fc-788e214bd627 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:34.893069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 533/1067] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:34.894348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:34.923062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:34.924915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:34.925200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:34.925479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:34.926219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:34.927600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:34.950224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 535/1068] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:38:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:38:35.019709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f56d-b630-4ee7-a298-88c4661add04 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:35.063157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:35.063785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.064059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.064327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.065025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:35.065327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:35.066737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:35.084278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 534/1069] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:35.086141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:35.152500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:35.201094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fb44f-8053-468a-a731-a3f2396e6433 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:35.234939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:35.235577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.235856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.236127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.242831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:35.244262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:35.255409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:35.256308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:35.257813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 536/1070] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:35.262822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:35.319945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:35.367782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64d987-6880-4adb-9356-3f18407b59b8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:35.402014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:35.403025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.403541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.404047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.405275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:35.405591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:35.407071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2338c-7e96-46eb-9c14-7a65b9581957 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:35.425869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 535/1071] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:35.428166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:35.538316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604566c-fa98-4b09-9b89-2894f529f4a0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:35.555158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:35.575019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:35.575620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.575904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.576182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/3cb6a7557a1e4eb99f23da0d8fea5536` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.578395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=8a4c233f6ff1442b8e08fbe6400ecf0d, role_id=3cb6a7557a1e4eb99f23da0d8fea5536)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:35.579870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037763ff-8fdf-4b8f-ad1f-55ac65121628 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:35.594137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 537/1072] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:35 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/8a4c233f6ff1442b8e08fbe6400ecf0d/roles/3cb6a7557a1e4eb99f23da0d8fea5536 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:35.900172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecf46-6c89-4c6f-9d3f-f30b89bf0dcd None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.900518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecf46-6c89-4c6f-9d3f-f30b89bf0dcd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.900694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecf46-6c89-4c6f-9d3f-f30b89bf0dcd None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.902071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 536/1073] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:35.907059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7a281-4054-4288-af31-384835fbad98 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:35.907488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7a281-4054-4288-af31-384835fbad98 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:35.907645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7a281-4054-4288-af31-384835fbad98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:35.925780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8be7a281-4054-4288-af31-384835fbad98 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:35.927010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8be7a281-4054-4288-af31-384835fbad98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:35.994097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 538/1074] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:36.015813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ecf46-6c89-4c6f-9d3f-f30b89bf0dcd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:36.130353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ecf46-6c89-4c6f-9d3f-f30b89bf0dcd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:36.145680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:36.168159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:36.168778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:36.169067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:36.169343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:36.179731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:36.184788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:36.185705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:36.186581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 537/1075] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:36.193637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7a281-4054-4288-af31-384835fbad98 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:36.292803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7a281-4054-4288-af31-384835fbad98 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:36.300191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:36.328837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:36.329454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:36.329732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:36.330032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:36.330738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:36.331040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:36.332385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:36.345980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 539/1076] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:36.347146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:36.447553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39bbe1-e659-486b-96e4-5e8ccb4efe95 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:36.481689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:36.482356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:36.482643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:36.482920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] PATH_INFO: `/v3/users/9afaf46b4a13420fa14c898baaa26712` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:36.484084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=9afaf46b4a13420fa14c898baaa26712)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:36.485509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:36.491771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 538/1077] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:36 2022] GET /identity/v3/users/9afaf46b4a13420fa14c898baaa26712 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:36.493500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:36.594454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ae6c3-76b3-4559-904e-6705808dfb21 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:36.629081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:36.629723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:36.630022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:36.630296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:36.631426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:36.632899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2d625-cc6e-4a1f-ad02-0e89231610a8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:36.639849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 540/1078] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:36 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:36.640989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:36.686381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:36.748296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed117261-83cb-4a13-9b2e-23a922899ab8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:36.782264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:36.782938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:36.783240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:36.783529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92/users/9afaf46b4a13420fa14c898baaa26712/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:36.785677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] RBAC: Authorizing `identity:create_grant(project_id=8e873b2933c64d74b86445d73388ad92, user_id=9afaf46b4a13420fa14c898baaa26712, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:36.787181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607801c1-39fd-447b-9a92-6160897cb02c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:36.803295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 539/1079] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:36 2022] PUT /identity/v3/projects/8e873b2933c64d74b86445d73388ad92/users/9afaf46b4a13420fa14c898baaa26712/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:37.043410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6504b-8661-451d-a380-ecd3538deab1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.043906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6504b-8661-451d-a380-ecd3538deab1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.043906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6504b-8661-451d-a380-ecd3538deab1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.045243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 541/1080] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:37.050463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.050757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.051037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.069550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:37.070790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:37.142140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 540/1081] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:37.165475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6504b-8661-451d-a380-ecd3538deab1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:37.202696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:37.271069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6504b-8661-451d-a380-ecd3538deab1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:37.305262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:37.305837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.306127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.306404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.374042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:37.375487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de12ca-efde-42d9-97c2-2a9a8d611801 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:37.410284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:37.478361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 542/1082] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:37.484405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:37.517298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:37.591098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac37f0-0e47-46ff-baa9-e57beb0d69ae None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:37.618507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832acdd-28c3-4ef8-96ce-17253dafdd84 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.618794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832acdd-28c3-4ef8-96ce-17253dafdd84 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.619084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832acdd-28c3-4ef8-96ce-17253dafdd84 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.620439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 543/1083] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:38:37.625362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.625675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.626094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.632146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:37.632795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.633605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.633924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.644775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:37.646118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:37.652433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:37.654339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:37.671241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:37.672138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:37.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:37.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:37.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:37.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:37.673031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 541/1084] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:37.678711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:37.724392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 544/1085] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:37.745593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:37.769539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:37.788053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8b540-7b5d-4d3a-9242-23183bafd2f5 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:37.820225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:37.825633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:37.826291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.826568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.826844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.827534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:37.827835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:37.829188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:37.842378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 542/1086] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:37.872910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f7bd4-2790-4efc-94e0-253412762040 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:37.912524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:37.913202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:37.913481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:37.913754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:37.926129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:37.931654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:37.932662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:37.938746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 545/1087] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 29 12:38:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:37.939816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.051415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc4024-5c1b-46d9-bb4b-d665ba2a27fd None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.087928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.088549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.088834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.089107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.089843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:38.090160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.091525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.100463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 543/1088] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:38:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:38.106821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.207820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbb9ad-7dd7-4fb8-808c-6d25820ce0b7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.232602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:38.241020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.241653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.241966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.242240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.253041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.254503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:38.268623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:38.269522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:38.270795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:38.270795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:38.270795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:38.270795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:38.270795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 546/1089] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:38.276242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.285627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:38.378622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b4c6d-5500-46cc-a565-1ebc61a28484 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.413729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.414398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.414677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.414948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.415632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:38.415934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.417310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.429112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 544/1090] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:38.435446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.534706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70ebe-a299-410d-b74e-37d03def5329 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.567546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.568179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.568456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.568728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.576709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.578164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:38:38.589468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:38.590642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:38.591638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 547/1091] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:38.597005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.662351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:38.698470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e0b4-2d44-4680-b3f9-492f8bcc0695 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.730795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.731422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.731705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.731981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.732772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:38.733080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.734551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8850ed-e255-413c-ba1e-f6ef4fd05d60 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.752239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 545/1092] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:38.757952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.798599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:38.816270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf7101-ff82-4b5e-9344-1b7f7eaf5e78 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.816564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf7101-ff82-4b5e-9344-1b7f7eaf5e78 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.816840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf7101-ff82-4b5e-9344-1b7f7eaf5e78 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.818232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 546/1093] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:38.823592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.823879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.824158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.842233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:38.843880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:38.860000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e117e-b983-49ad-9402-72d8f25c4d68 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:38.893747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:38.894394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:38.894674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:38.894948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:38.910643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 547/1094] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:38:38.934952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:38.962458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:38.963924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:38.989053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.002754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.035874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5ff43-a3d0-44f4-81b4-76af0be01088 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.063974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 548/1095] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 12:38:39.069601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.069766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.070270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.071092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.071368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.081544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:39.086425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:39.087541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:38:39.088734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 548/1096] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:39.093632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.176653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0c71-4098-4e27-8592-23b8f9641f9d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.192129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6caa7-dc35-4119-b64d-8a947bb33f35 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.205804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.210479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.211661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.211910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.212181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.220833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.227587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.229039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.230652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.231586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.232022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.232450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.233496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:39.233808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.235235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:38:39.244491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:39.245473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:39.246412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:38:39.246412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:38:39.246412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:38:39.246412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:39.246638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 549/1097] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.246721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 549/1098] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:39.250914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.251051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.348882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72f322-5ee6-47a8-85a1-206ee5314986 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.349565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda60b-962a-4d49-b00a-a1cf48c823a0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.381974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.382404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.382581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.382864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.383079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.383756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.383893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.384171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] PATH_INFO: `/v3/users/9afaf46b4a13420fa14c898baaa26712` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.384449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:39.384757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.385341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] RBAC: Authorizing `identity:get_user(user_id=9afaf46b4a13420fa14c898baaa26712)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.386132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.386820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.388422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 550/1099] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/users/9afaf46b4a13420fa14c898baaa26712 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.394023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.398182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 550/1100] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.403619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.439136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.452286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.494659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4feedd-f48d-4a47-b065-b4de1ef8fb68 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.502239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f38ca-d15d-4ba7-86fb-6a94e435d2c1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.528117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.528734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.529014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.529285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.538444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.539082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.539447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.539730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.546302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.547765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:38:39.558962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:39.559839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:39.560956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 551/1101] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:38:39.566446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.598128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.599560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.642018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.663968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73dac-dac6-4dcb-b6e3-b5becda5d4f9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.674929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.697043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 551/1102] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 12:38:39.699098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.699799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.700088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.700366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.701112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:39.701424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.702951 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.703076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.718112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 552/1103] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.722328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.808772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a49d1-f428-4864-9050-e01f85cdf66a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.823172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e84b8-988c-4d20-ae2e-937e611dc08b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:39.842014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.842661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.843206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.843481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.844628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.846075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.847727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 552/1104] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.854228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:39.856700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:39.857314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:39.857597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:39.857897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:39.858682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] RBAC: Adding query filter params (role.id=3cb6a7557a1e4eb99f23da0d8fea5536, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=8a4c233f6ff1442b8e08fbe6400ecf0d) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:39.858991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:39.860405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ab54b-4d46-4e62-8ce5-04111169c062 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:39.872657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 553/1105] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/role_assignments?user.id=8a4c233f6ff1442b8e08fbe6400ecf0d&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=3cb6a7557a1e4eb99f23da0d8fea5536 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:39.875660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:39.968297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5a9f-6053-4b73-9cd0-46338e976964 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:40.002502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:40.003138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:40.003419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:40.003692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:40.004493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=8e873b2933c64d74b86445d73388ad92, user.id=9afaf46b4a13420fa14c898baaa26712) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:40.004807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:40.006266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaaa92-a803-4a91-a11a-66940de56f9a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:40.012584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 553/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:39 2022] GET /identity/v3/role_assignments?user.id=9afaf46b4a13420fa14c898baaa26712&scope.project.id=8e873b2933c64d74b86445d73388ad92&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:38:42.051091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89f4c-eb5a-451c-bac5-21eb8678f30f None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:42.051640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89f4c-eb5a-451c-bac5-21eb8678f30f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:42.051640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f89f4c-eb5a-451c-bac5-21eb8678f30f None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:42.052972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 554/1107] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:42.058861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:42.059266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:42.059421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:42.077045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:42.078253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:42.143163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 554/1108] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:42.167759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f89f4c-eb5a-451c-bac5-21eb8678f30f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:42.201880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:42.268647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f89f4c-eb5a-451c-bac5-21eb8678f30f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:42.300797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:42.301472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:42.301705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:42.302011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:42.303152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:42.304596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b488e0f-d7c4-4bb8-bd79-4161f8ea1854 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:42.306733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 555/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:42.312470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:42.321207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:42.410459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020750fb-4070-4271-bf40-bc532b056a9a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:42.446718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:42.447336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:42.447614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:42.447886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:42.448579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:42.450031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776432e-65fe-43bc-a365-d141c716cdb1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:42.477398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 555/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:38:42 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:44.540514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24438999-e2b5-4203-a5fc-7dc8e33f7f15 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:44.541096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24438999-e2b5-4203-a5fc-7dc8e33f7f15 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:44.541778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24438999-e2b5-4203-a5fc-7dc8e33f7f15 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:44.543162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 556/1111] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:44.548329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:44.548780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:44.548882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:44.567012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:44.568192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:44.623104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:44.634523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 556/1112] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:44.657401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24438999-e2b5-4203-a5fc-7dc8e33f7f15 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:44.756782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24438999-e2b5-4203-a5fc-7dc8e33f7f15 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:44.758346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:44.791211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:44.791778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:44.792061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:44.792338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] PATH_INFO: `/v3/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:44.793795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] RBAC: Authorizing `identity:get_role(role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:44.795185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:44.796503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 557/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:44 2022] GET /identity/v3/roles/6dda70e53deb474a96e637c5fff03848 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:44.802266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:44.910777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f03df4-a213-4623-8280-7e756404e7a0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:44.944075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:44.944709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:44.944994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:44.945796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] PATH_INFO: `/v3/users/27201a81abf240ef9dcc29b36d1807c1` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:44.958904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=27201a81abf240ef9dcc29b36d1807c1)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:44.960341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:44.962681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 557/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:44 2022] GET /identity/v3/users/27201a81abf240ef9dcc29b36d1807c1 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:44.968800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:45.065246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4537-8cb1-4747-8d93-c9dc5af5eb3b None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:45.099039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:45.099655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:45.099932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:45.100208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:45.137993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:45.166320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:45.167782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:45.264450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 558/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:45.270394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:45.319800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:45.369328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e17a-6ed1-4f35-b7e2-ff1ea01d9aa3 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:45.403682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:45.404315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:45.404603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:45.404878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:45.406057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:45.407490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c056-2121-4283-94f6-de92e22e0035 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:45.409143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 558/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:45 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:45.415773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:45.482762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:45.513895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a93941-0cd8-40d7-82c3-283f337c3b6c None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:45.548352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:45.548965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:45.549246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:45.549523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:45.550332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=8e873b2933c64d74b86445d73388ad92, user.id=27201a81abf240ef9dcc29b36d1807c1) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:45.550646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:45.552064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19df16a-e054-4c05-b713-f7877c55b66f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:45.557390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 559/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:45 2022] GET /identity/v3/role_assignments?user.id=27201a81abf240ef9dcc29b36d1807c1&scope.project.id=8e873b2933c64d74b86445d73388ad92&role.id=6dda70e53deb474a96e637c5fff03848 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:47.534199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376275-5435-4b1a-a822-3f99fc7948e7 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:47.534706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376275-5435-4b1a-a822-3f99fc7948e7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:47.534706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376275-5435-4b1a-a822-3f99fc7948e7 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:47.536022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 559/1118] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:47.541894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:47.542318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:47.542477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:47.561140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:47.562407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:47.634913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 560/1119] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:47.658115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f376275-5435-4b1a-a822-3f99fc7948e7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:47.758475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f376275-5435-4b1a-a822-3f99fc7948e7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:47.775014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:47.791870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:47.792433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:47.792715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:47.792985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] PATH_INFO: `/v3/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:47.794438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] RBAC: Authorizing `identity:get_role(role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:47.795791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:47.797105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 560/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:47 2022] GET /identity/v3/roles/6dda70e53deb474a96e637c5fff03848 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:47.802812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:47.863624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:47.907500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5ffc0-fafd-46c9-bda1-9beff90183c8 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:47.940985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:47.941621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:47.941932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:47.942208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] PATH_INFO: `/v3/users/27201a81abf240ef9dcc29b36d1807c1` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:47.943351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] RBAC: Authorizing `identity:get_user(user_id=27201a81abf240ef9dcc29b36d1807c1)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:47.944770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:47.946385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 561/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:47 2022] GET /identity/v3/users/27201a81abf240ef9dcc29b36d1807c1 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:47.952129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:48.050795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20fd6-8087-4fe5-8173-de2772453f6c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:48.083616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:48.084237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:48.084514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:48.084785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:48.085957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:48.087375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f02cf7-dc50-49a5-90df-98d4e6eaea2d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:48.089007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 561/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:47 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:48.095306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:48.192770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aad73-3ede-4360-a53c-8813513e7f34 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:48.222034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:48.227884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:48.228542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:48.228827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:48.229104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92/users/27201a81abf240ef9dcc29b36d1807c1/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:48.231389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] RBAC: Authorizing `identity:create_grant(project_id=8e873b2933c64d74b86445d73388ad92, user_id=27201a81abf240ef9dcc29b36d1807c1, role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:48.232816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043783d3-1542-47be-b3e6-0d676385e78c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:48.245364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 562/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:38:48 2022] PUT /identity/v3/projects/8e873b2933c64d74b86445d73388ad92/users/27201a81abf240ef9dcc29b36d1807c1/roles/6dda70e53deb474a96e637c5fff03848 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:38:50.191249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410a53-fe7d-4a59-91d4-d4c23b2c3988 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:50.191757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410a53-fe7d-4a59-91d4-d4c23b2c3988 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:50.191859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410a53-fe7d-4a59-91d4-d4c23b2c3988 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:50.193156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 562/1124] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:50.198203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:50.198593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:50.198756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:50.216270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:50.217428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:50.281732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 563/1125] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:50.305175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd410a53-fe7d-4a59-91d4-d4c23b2c3988 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:50.360902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:50.405589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd410a53-fe7d-4a59-91d4-d4c23b2c3988 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:50.438253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:50.438878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:50.439160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:50.439436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] PATH_INFO: `/v3/roles/6dda70e53deb474a96e637c5fff03848` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:50.440870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] RBAC: Authorizing `identity:get_role(role_id=6dda70e53deb474a96e637c5fff03848)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:50.442273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:50.443591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 563/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:50 2022] GET /identity/v3/roles/6dda70e53deb474a96e637c5fff03848 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:50.449225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:50.544923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519e5a-10b6-4e77-ad41-47b2967955aa None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:50.566428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:50.578355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:50.578986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:50.579267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:50.579540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] PATH_INFO: `/v3/users/27201a81abf240ef9dcc29b36d1807c1` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:50.580739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] RBAC: Authorizing `identity:get_user(user_id=27201a81abf240ef9dcc29b36d1807c1)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:50.582215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:50.583801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 564/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:50 2022] GET /identity/v3/users/27201a81abf240ef9dcc29b36d1807c1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:50.589978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:50.686851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650596f2-0c6a-4880-9ca2-039c9dbd4857 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:50.708008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:50.720494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:50.721098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:50.721382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:50.721660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:50.787166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:50.788629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:50.884569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 564/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:50.890510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:50.990330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10b478-6981-42f4-80f9-c8b3f6adfc46 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:51.024697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:51.025332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:51.025614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:51.025919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] PATH_INFO: `/v3/projects/8e873b2933c64d74b86445d73388ad92` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:51.027063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] RBAC: Authorizing `identity:get_project(project_id=8e873b2933c64d74b86445d73388ad92)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:51.028488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fdb4c-6945-4ea3-b956-d14216bca057 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:51.030180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 565/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:50 2022] GET /identity/v3/projects/8e873b2933c64d74b86445d73388ad92 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:51.036282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:51.059788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:51.135859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba281a5-f38b-4e00-b34e-4f94938811c8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:51.169175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:51.169779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:51.170086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:51.170356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:51.171135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] RBAC: Adding query filter params (role.id=6dda70e53deb474a96e637c5fff03848, scope.project.id=8e873b2933c64d74b86445d73388ad92, user.id=27201a81abf240ef9dcc29b36d1807c1) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:51.171467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:51.172880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05070d-0eb4-4af2-add1-3bd6399b4318 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:51.178570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 565/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:51 2022] GET /identity/v3/role_assignments?user.id=27201a81abf240ef9dcc29b36d1807c1&scope.project.id=8e873b2933c64d74b86445d73388ad92&role.id=6dda70e53deb474a96e637c5fff03848 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:53.219005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb82f-e611-475d-9bfe-871d0e87dbd9 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:53.219505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb82f-e611-475d-9bfe-871d0e87dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:53.219612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb82f-e611-475d-9bfe-871d0e87dbd9 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:53.220879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 566/1131] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:53.226187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:53.226619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:53.226745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:53.244397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:53.245570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:53.314478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 566/1132] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:53.338578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb82f-e611-475d-9bfe-871d0e87dbd9 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:53.396001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:53.437831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb82f-e611-475d-9bfe-871d0e87dbd9 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:53.470812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:53.471432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:53.471715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:53.471988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:53.473136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:53.474593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04c9d1-9247-4e92-8b05-bd5343af9787 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:53.476211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 567/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:53.481954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:53.499495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:53.579472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a57df-dc8b-4d63-99f4-fb95c6f6f63d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:53.612381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:53.612983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:53.613263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:53.613536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:53.614248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:53.615666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed8b52-3971-45cc-ac26-4d932b8b46dc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:53.641612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 567/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:38:53 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:55.685016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166759-86b6-4a85-aa61-54851182a00b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:55.685560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166759-86b6-4a85-aa61-54851182a00b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:55.685560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166759-86b6-4a85-aa61-54851182a00b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:55.686991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 568/1135] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:55.692210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:55.692642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:55.692750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:55.711525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:55.712766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:55.772717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:55.781848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 568/1136] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:55.807628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee166759-86b6-4a85-aa61-54851182a00b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:55.906433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee166759-86b6-4a85-aa61-54851182a00b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:55.931368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:55.940355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:55.940927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:55.941211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:55.941492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:55.942669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:55.944102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff1253-cf3d-486c-8f02-df9687d1092a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:55.946327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 569/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:38:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:55.951944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:56.047565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae430-a551-4641-b88e-8a8ced78ae60 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:56.080389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:56.080996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:56.081279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:56.081555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:56.082255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:56.083685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e11997-48a6-470f-8e55-1e0e5355e8be None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:56.106100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 569/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:38:55 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:38:58.145086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5faa-c972-4b2c-b257-45f0dcc2248c None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.145520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5faa-c972-4b2c-b257-45f0dcc2248c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.145632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5faa-c972-4b2c-b257-45f0dcc2248c None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.147034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 570/1139] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:38:58.152131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.152540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.152687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.170489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:38:58.171666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:38:58.220908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:58.239676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 570/1140] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:38:58.263273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc5faa-c972-4b2c-b257-45f0dcc2248c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:58.377361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc5faa-c972-4b2c-b257-45f0dcc2248c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:58.410744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:58.411377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.411662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.411941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.421794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:38:58.427012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:38:58.428151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:38:58.429346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 571/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:38:58.434364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:58.529458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873aa64-a164-4df3-977b-e66e01ec4e2f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:58.561502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:58.562134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.562413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.562683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.563386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:58.563688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:58.565068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:58.573302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 571/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:58.579036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:58.637777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:58.697640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccef3d3-3166-4963-91b6-5bacdf337058 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:58.733275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:58.733939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.734223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.734499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] PATH_INFO: `/v3/users/98007819302e46509114470051ced413` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.746956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] RBAC: Authorizing `identity:get_user(user_id=98007819302e46509114470051ced413)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:58.748383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:58.750021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 572/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:38:58 2022] GET /identity/v3/users/98007819302e46509114470051ced413 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:58.755813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:58.802259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:58.854446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01918d-945f-4f7a-8a06-27c28eca931a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:58.886115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:58.886712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:58.886989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:58.887260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:58.952529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:58.953920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:59.008133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:59.061590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 572/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:38:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:38:59.067740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:59.164933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120f699-377d-4961-95a6-1f64c18d5e0d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:59.178593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:38:59.200658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:59.201290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:59.201570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:59.201859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] PATH_INFO: `/v3/projects/a35b0febcd394a93889b737561121620` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:59.203048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] RBAC: Authorizing `identity:get_project(project_id=a35b0febcd394a93889b737561121620)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:59.204418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20025cfd-ae07-4f06-88d4-a539cee04a5e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:59.206094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 573/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:38:59 2022] GET /identity/v3/projects/a35b0febcd394a93889b737561121620 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:38:59.212962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:38:59.308781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404af96-2dfe-400a-8103-be28fc129c89 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:38:59.342390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:38:59.343282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:38:59.343559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:38:59.343830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:38:59.344619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=a35b0febcd394a93889b737561121620, user.id=98007819302e46509114470051ced413) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:38:59.344926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:38:59.346394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5020ad-3251-4cb3-9965-886b8978148f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:38:59.352225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 573/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:38:59 2022] GET /identity/v3/role_assignments?user.id=98007819302e46509114470051ced413&scope.project.id=a35b0febcd394a93889b737561121620&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:01.298147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e7744-35e0-40e6-ae8a-11e3027a7921 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.298659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e7744-35e0-40e6-ae8a-11e3027a7921 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.298768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e7744-35e0-40e6-ae8a-11e3027a7921 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.300048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 574/1147] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:01.305371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.305786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.305963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.324485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:01.325658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:01.328407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:01.390738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 574/1148] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:01.413802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e7744-35e0-40e6-ae8a-11e3027a7921 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:01.513241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e7744-35e0-40e6-ae8a-11e3027a7921 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:01.545994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:01.546617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.546897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.547170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.556894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:39:01.561688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:01.562808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:39:01.563817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 575/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:01.568671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:01.664270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc393-2d69-4069-bde7-b895f6e01aed None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:01.696658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:01.697325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.697605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.697921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.698612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:01.698914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:01.700265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:01.708077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 575/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:01.714439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:01.757129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:01.813975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b932f-7e45-41ab-8b89-f7a02ba000f3 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:01.847624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:01.848248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.848532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.848810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] PATH_INFO: `/v3/users/98007819302e46509114470051ced413` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.849988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] RBAC: Authorizing `identity:get_user(user_id=98007819302e46509114470051ced413)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:01.851491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:01.853068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 576/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:39:01 2022] GET /identity/v3/users/98007819302e46509114470051ced413 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:01.858697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:01.861487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:01.957264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279c10d-1629-43cf-86cc-03b181ef4219 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:01.989102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:01.989728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:01.990036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:01.990311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] PATH_INFO: `/v3/projects/a35b0febcd394a93889b737561121620` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:01.991975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] RBAC: Authorizing `identity:get_project(project_id=a35b0febcd394a93889b737561121620)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:01.993392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d3f64-fd6a-43b2-85d0-a8225f39c401 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:01.995051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 576/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:39:01 2022] GET /identity/v3/projects/a35b0febcd394a93889b737561121620 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:02.001004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:02.097174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7ae66-34c1-4f45-b122-bc431cd2898f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:02.106689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:02.131720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:02.132416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:02.132698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:02.132975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] PATH_INFO: `/v3/projects/a35b0febcd394a93889b737561121620/users/98007819302e46509114470051ced413/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:02.134919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] RBAC: Authorizing `identity:create_grant(project_id=a35b0febcd394a93889b737561121620, user_id=98007819302e46509114470051ced413, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:02.136342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74d8e2-45a9-4df6-bccf-9210dcbb3860 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:02.150141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 577/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:39:02 2022] PUT /identity/v3/projects/a35b0febcd394a93889b737561121620/users/98007819302e46509114470051ced413/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:39:04.168738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13587a4-6653-4afb-a861-628371d824eb None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.169224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13587a4-6653-4afb-a861-628371d824eb None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.169334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13587a4-6653-4afb-a861-628371d824eb None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.170665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 577/1154] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:04.175665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.176050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.176252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.195436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:04.196688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:04.262206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 578/1155] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:04.286617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13587a4-6653-4afb-a861-628371d824eb None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:04.354762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:04.387396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13587a4-6653-4afb-a861-628371d824eb None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:04.419420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:04.420036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.420312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.420579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.430269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:39:04.435081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:04.436531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:39:04.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 578/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:04.442352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:04.534727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:04.538691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a8166-f939-4e9f-abac-f65674465575 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:04.573114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:04.573730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.574042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.574322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.575034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:04.575340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:04.576706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:04.584508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 579/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:04.590923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:04.686394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1e5f-e4a1-4ba6-9b79-88fafad94fb0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:04.713758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:04.720072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:04.720701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.720977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.721246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] PATH_INFO: `/v3/users/98007819302e46509114470051ced413` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.722410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] RBAC: Authorizing `identity:get_user(user_id=98007819302e46509114470051ced413)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:04.723816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:04.725396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 579/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:39:04 2022] GET /identity/v3/users/98007819302e46509114470051ced413 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:04.731770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:04.827708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb7db-e6e6-437f-ace1-747271a539fe None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:04.860594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:04.861195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:04.861476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:04.861753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:04.887690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:04.926617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:04.928044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:05.021582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 580/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:39:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:39:05.027325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:05.124395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1dd9c-f81e-4c9e-ad8e-fa9f04450380 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:05.157211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:05.157863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:05.158153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:05.158433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] PATH_INFO: `/v3/projects/a35b0febcd394a93889b737561121620` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:05.159582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] RBAC: Authorizing `identity:get_project(project_id=a35b0febcd394a93889b737561121620)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:05.161002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa760f32-a08c-4897-ab96-6866af0fd062 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:05.162689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 580/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:39:05 2022] GET /identity/v3/projects/a35b0febcd394a93889b737561121620 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:05.169323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:05.225414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:05.265738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4dc4-1eb5-4b7e-a998-b2df9bce3d2f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:05.299174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:05.299710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:05.299990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:05.300262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:05.301039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=a35b0febcd394a93889b737561121620, user.id=98007819302e46509114470051ced413) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:05.301346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:05.302788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7f529-9311-4d3c-8750-7dbee6429d46 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:05.309228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 581/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:39:05 2022] GET /identity/v3/role_assignments?user.id=98007819302e46509114470051ced413&scope.project.id=a35b0febcd394a93889b737561121620&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:07.288695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6486-89ac-4b27-88b0-7cf6608260b6 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:07.289217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6486-89ac-4b27-88b0-7cf6608260b6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:07.289345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6486-89ac-4b27-88b0-7cf6608260b6 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:07.290750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 581/1162] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:07.295932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:07.296400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:07.296502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:07.315749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:07.316927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:07.380869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 582/1163] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:07.403507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6486-89ac-4b27-88b0-7cf6608260b6 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:07.468316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:07.502482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6486-89ac-4b27-88b0-7cf6608260b6 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:07.535078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:07.535688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:07.536004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:07.536281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:07.604462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:07.605959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:07.666603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:07.701644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 582/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:39:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:39:07.707707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:07.761581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:07.804636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d451-43f7-42f5-a3d7-7d688cf6aac0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:07.838142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:07.838778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:07.839064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:07.839339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:07.846428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:07.847859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-6e3ade27-d50a-4121-80e3-cdd5b27e0e1a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:07.859703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:07.860920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:39:07.862143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 583/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:39:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:07.866889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:07.977352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfe9de-19b8-4d1f-9b81-45ac76c6a55c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:08.010913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:08.011550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:08.011847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:08.012130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:08.012821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:08.013128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:08.014518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97034c-8cf8-451e-9a7a-d32fbc0b77c0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:08.021475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 583/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 29 12:39:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:09.962181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3e95-1d21-440e-9448-babe4f7934c4 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:09.962769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3e95-1d21-440e-9448-babe4f7934c4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:09.962769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3e95-1d21-440e-9448-babe4f7934c4 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:09.964122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 584/1167] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:09.969369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f748-8c09-405c-9962-42769216a180 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:09.969744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f748-8c09-405c-9962-42769216a180 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:09.969956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f748-8c09-405c-9962-42769216a180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:09.988025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f0a3f748-8c09-405c-9962-42769216a180 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:09.989185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f0a3f748-8c09-405c-9962-42769216a180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:10.011593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:10.053232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 584/1168] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:10.076192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf3e95-1d21-440e-9448-babe4f7934c4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:10.176466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf3e95-1d21-440e-9448-babe4f7934c4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:10.190073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:10.211294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:10.211913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:10.212244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:10.212474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:10.213192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:10.214585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff24c2-da92-48c3-b331-bac01acb2750 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:10.229449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 585/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Fri Jul 29 12:39:10 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:39:12.396013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347102b-8119-4491-83a4-5563e29f4066 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:12.396532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347102b-8119-4491-83a4-5563e29f4066 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:12.396532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347102b-8119-4491-83a4-5563e29f4066 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:12.398406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 585/1170] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:12.403489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:12.403862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:12.404054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:12.422462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:12.423657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:12.490640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 586/1171] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:12.512996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347102b-8119-4491-83a4-5563e29f4066 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:12.611722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347102b-8119-4491-83a4-5563e29f4066 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:12.644243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:12.644865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:12.645143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:12.645415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] PATH_INFO: `/v3/domains/5b0cf86787b94cfc84f677b58e10e846` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:12.656016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b0cf86787b94cfc84f677b58e10e846)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:12.657381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a52bf-44d2-4248-8225-ab2148fddf36 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:12.658989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 586/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Fri Jul 29 12:39:12 2022] GET /identity/v3/domains/5b0cf86787b94cfc84f677b58e10e846 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:12.665152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:12.761145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160241d8-3950-4f58-96a3-fe82072ad705 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:12.763429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:12.796811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:12.797429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:12.797712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:12.798020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:12.798700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:12.800128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9b42-4a80-4a51-b148-073dc9f3b67c None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:12.826168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 587/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:39:12 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:39:14.886665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef18945-581a-4d45-84d5-12530a4e3143 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:14.887179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef18945-581a-4d45-84d5-12530a4e3143 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:14.887302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef18945-581a-4d45-84d5-12530a4e3143 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:14.888554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 587/1174] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:14.893627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:14.894084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:14.894201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:14.912580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:14.913757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:14.978636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 588/1175] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:15.002801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef18945-581a-4d45-84d5-12530a4e3143 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:15.025793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:15.104394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef18945-581a-4d45-84d5-12530a4e3143 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:15.137759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:15.138362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:15.138641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:15.138913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] PATH_INFO: `/v3/domains/5b0cf86787b94cfc84f677b58e10e846` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:15.140051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b0cf86787b94cfc84f677b58e10e846)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:15.141464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b498c5-aa09-42be-a886-0b2d774fefd4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:15.143079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 588/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Fri Jul 29 12:39:15 2022] GET /identity/v3/domains/5b0cf86787b94cfc84f677b58e10e846 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:15.148753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:15.230683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:15.246370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4a9a-8cfa-431d-ac9d-c70a0d67a662 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:15.278803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:15.279407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:15.279685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:15.279957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:15.280634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:15.282079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b2baf-bf67-4e23-b563-6156b46215a2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:15.302196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 589/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:39:15 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:39:17.345113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde7ce-ac48-47c7-9f5c-dc124fc2b438 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:17.345660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde7ce-ac48-47c7-9f5c-dc124fc2b438 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:17.345660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde7ce-ac48-47c7-9f5c-dc124fc2b438 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:17.347044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 589/1178] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:17.352284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:17.352674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:17.352843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:17.374226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:17.375350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:17.445632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 590/1179] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:17.468897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fde7ce-ac48-47c7-9f5c-dc124fc2b438 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:17.557780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:17.571660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fde7ce-ac48-47c7-9f5c-dc124fc2b438 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:17.604900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:17.605523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:17.605806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:17.606111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:17.616508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:39:17.621365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:17.622469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:39:17.623378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 590/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:17.628464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:17.697181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:17.727406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5f866-9604-40a0-b73d-e9280cd8991e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:17.759442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:17.760049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:17.760330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:17.760605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:17.761319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:17.761624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:17.763017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:17.770925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 591/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:17.777183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:17.875983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c0018-98d2-47d3-bced-e6c63dd9416d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:17.909712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:17.910301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:17.910586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:17.910859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] PATH_INFO: `/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:17.923934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] RBAC: Authorizing `identity:get_user(user_id=fb603b91b8c546f3b8e0e8dd176b7f33)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:17.925362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:17.926977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 591/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:39:17 2022] GET /identity/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:17.933256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:18.029483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36fc1-f5a0-4ec6-b65d-f5a0d58ec023 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:18.061861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:18.062493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:18.062799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:18.063099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:18.065614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:18.131606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:18.133085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:18.228284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 592/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:39:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:39:18.234046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:18.243257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:18.345896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c224-8f08-4ed4-b0fa-f7d1fb8915f8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:18.378475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:18.379113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:18.379434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:18.379717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] PATH_INFO: `/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:18.380871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] RBAC: Authorizing `identity:get_project(project_id=a5ad17a75a8f41a29b1a934ed8c0dd7f)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:18.382947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3993419-d4e9-46ff-ac74-8141dcc1a9b8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:18.384616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 592/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:39:18 2022] GET /identity/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:18.390774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:18.423005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:18.488038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7421bbe-1316-45af-8254-2e15e3bc0d76 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:18.521008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:18.521615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:18.522454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:18.522734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:18.523518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=a5ad17a75a8f41a29b1a934ed8c0dd7f, user.id=fb603b91b8c546f3b8e0e8dd176b7f33) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:18.523830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:18.525249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab7e43-3d9b-4d40-8014-629adf279f72 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:18.530564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 593/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:39:18 2022] GET /identity/v3/role_assignments?user.id=fb603b91b8c546f3b8e0e8dd176b7f33&scope.project.id=a5ad17a75a8f41a29b1a934ed8c0dd7f&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:20.515573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986107-a1eb-42bd-82ee-a0c1fe844146 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:20.516139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986107-a1eb-42bd-82ee-a0c1fe844146 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:20.516139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986107-a1eb-42bd-82ee-a0c1fe844146 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:20.517392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 593/1186] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:20.523120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:20.523502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:20.523675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:20.541249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:20.542519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:20.607941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 594/1187] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:39:20.630508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f986107-a1eb-42bd-82ee-a0c1fe844146 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:20.705710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:20.732364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f986107-a1eb-42bd-82ee-a0c1fe844146 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:20.765995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:20.766609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:20.766910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:20.767210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:20.777784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:39:20.782750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:20.783892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:39:20.784992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 594/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:20.790607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:20.817015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:20.887239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587e6c8-02f1-4b00-aa5e-99a007060d53 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:20.921046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:20.921661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:20.921978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:20.922255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:20.922969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:20.923279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:20.924633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:20.932563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 595/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:20.938344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:21.045124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1878-5a77-487e-866a-effa3dd0ba3c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:21.078583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:21.079216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:21.079500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:21.079775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] PATH_INFO: `/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:21.080933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] RBAC: Authorizing `identity:get_user(user_id=fb603b91b8c546f3b8e0e8dd176b7f33)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:21.082412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:21.084061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 595/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:39:20 2022] GET /identity/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:21.089975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:21.177936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:21.188118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b6aa3-7cad-46a8-829d-d54c05b23bb5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:21.220573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:21.221199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:21.221476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:21.221746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] PATH_INFO: `/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:21.222916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] RBAC: Authorizing `identity:get_project(project_id=a5ad17a75a8f41a29b1a934ed8c0dd7f)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:21.224326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2612a-d348-4cb5-8d78-922e046c8f6e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:21.226018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 596/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:39:21 2022] GET /identity/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:21.231965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:21.234876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:21.331622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e2e03-04f0-4a29-8cf1-4e92e06f1d6c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:21.364401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:21.365085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:21.365367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:21.366209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] PATH_INFO: `/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f/users/fb603b91b8c546f3b8e0e8dd176b7f33/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:21.368256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] RBAC: Authorizing `identity:create_grant(project_id=a5ad17a75a8f41a29b1a934ed8c0dd7f, user_id=fb603b91b8c546f3b8e0e8dd176b7f33, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:21.369691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0fc2-53da-455e-8efd-99fa394ee1ac None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:21.384208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 596/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:39:21 2022] PUT /identity/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f/users/fb603b91b8c546f3b8e0e8dd176b7f33/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:39:23.346418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b713-2f00-4c12-9e16-4e1c4e0da792 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:23.346952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b713-2f00-4c12-9e16-4e1c4e0da792 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:23.346952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3b713-2f00-4c12-9e16-4e1c4e0da792 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:23.348337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 597/1193] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:39:23.354183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:23.354590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:23.354738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:23.373002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:39:23.374145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:39:23.434472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:23.449842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 597/1194] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:39:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:39:23.473141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3b713-2f00-4c12-9e16-4e1c4e0da792 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:23.571811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3b713-2f00-4c12-9e16-4e1c4e0da792 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:23.604743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:23.605360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:23.605638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:23.605938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:23.616059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:39:23.620826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:39:23.621986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:39:23.623078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 598/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:39:23.628246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:23.725630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d12ba-5e49-4e5c-a4ba-02c472cf9bb7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:23.758638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:23.759255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:23.759539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:23.759819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:23.760538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:23.760848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:23.762266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:23.780467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 598/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:23.786294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:23.805893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:23.884406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52511a-046e-4b02-891a-f6e41d731d29 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:23.916497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:23.917120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:23.917400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:23.917673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] PATH_INFO: `/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:23.918836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=fb603b91b8c546f3b8e0e8dd176b7f33)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:23.920261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:23.921842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 599/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 29 12:39:23 2022] GET /identity/v3/users/fb603b91b8c546f3b8e0e8dd176b7f33 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:23.927472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:23.982254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:24.026511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a51f-4747-4ca7-b2c8-1b9b55cc438b None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:24.058576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:24.059182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:24.059465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:24.059740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:24.126110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:24.127558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:24.179788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:24.224427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 599/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:39:24.230340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:24.311305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:24.326448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe881c-00b6-48a8-92fd-7f52320cb9d1 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:24.358681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:24.359305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:24.359584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:24.359859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] PATH_INFO: `/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:24.360997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] RBAC: Authorizing `identity:get_project(project_id=a5ad17a75a8f41a29b1a934ed8c0dd7f)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:24.362441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05249a-5c7a-4939-9e7c-ab537341ca61 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:24.364061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 600/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 29 12:39:24 2022] GET /identity/v3/projects/a5ad17a75a8f41a29b1a934ed8c0dd7f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:24.370522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:39:24.466984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d11358-ca13-425c-bcd6-be9f4caa386a None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:39:24.500314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:39:24.500905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:39:24.501188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:39:24.501457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:39:24.502257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=a5ad17a75a8f41a29b1a934ed8c0dd7f, user.id=fb603b91b8c546f3b8e0e8dd176b7f33) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:39:24.502563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:39:24.503955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0a79b-e5cd-4195-9f34-6f5e8c2060ce None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:39:24.509493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 600/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:39:24 2022] GET /identity/v3/role_assignments?user.id=fb603b91b8c546f3b8e0e8dd176b7f33&scope.project.id=a5ad17a75a8f41a29b1a934ed8c0dd7f&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:39:34.354693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=23 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:39:34.507107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:19.787675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6d21-f64f-4a42-a2d4-a6d112f98e91 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:19.788402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6d21-f64f-4a42-a2d4-a6d112f98e91 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:19.788402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6d21-f64f-4a42-a2d4-a6d112f98e91 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:19.790569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 601/1201] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:40:19.797481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:19.798018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:19.798101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:19.820511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:19.821927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:19.903700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 601/1202] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:19.931240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6d21-f64f-4a42-a2d4-a6d112f98e91 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:20.068976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6d21-f64f-4a42-a2d4-a6d112f98e91 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:20.115394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:20.116095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:20.116422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:20.116702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:20.130686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:20.132700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:20.149873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:20.150967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:20.154988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:20.154988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:20.154988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:20.154988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:20.154988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 602/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:20.158893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:20.286644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f711d8-62e7-47eb-bd4c-b211b3f146d7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:20.331005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:20.338751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:20.339456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:20.339741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:20.340033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:20.340772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:20.341061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:20.342787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d3a8-108c-45c1-a9ed-f41564e35782 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:20.362002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 602/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:20.366628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:20.446760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:20.508938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb891a-1f32-41df-92d9-11eeea684687 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:20.554711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:20.555465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:20.555848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:20.556135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:20.556938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:20.558567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac1db2-eff9-44b6-bcc8-cd09f1afd227 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:20.590099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 603/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 29 12:40:20 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:40:22.821611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dde01-f63a-459b-a7ef-fb3b3b597d8b None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:22.822212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dde01-f63a-459b-a7ef-fb3b3b597d8b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:22.822212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dde01-f63a-459b-a7ef-fb3b3b597d8b None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:22.823747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 603/1206] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:22.828870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:22.829411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:22.829411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:22.850725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:22.851926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:22.928731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 604/1207] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:22.951438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dde01-f63a-459b-a7ef-fb3b3b597d8b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:23.069615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dde01-f63a-459b-a7ef-fb3b3b597d8b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:23.105974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:23.106577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:23.106877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:23.107158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:23.119304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:23.124328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:23.125329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:23.126299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 604/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:23.133519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:23.177359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:23.248089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddd549-3fdf-4936-a52e-46a0d9150d70 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:23.288750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:23.289419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:23.289637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:23.289942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:23.290678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:23.290980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:23.292368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:23.301566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 605/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:40:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:23.307759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:23.375388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:23.432272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c5f4f-866c-4f01-94fa-4bd463b3ada7 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:23.486799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:23.487598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:23.488039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:23.488447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:23.502657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:23.504859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:23.522528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:23.523692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:23.524749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:23.524749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:23.524749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:23.524749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:23.524749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 605/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:23.530501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:23.647537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:23.650106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001bad-7747-4737-9f92-0da1e90daeea None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:23.692153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:23.692835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:23.693192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:23.693544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:23.694452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:23.694847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:23.696368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:23.712373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 606/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:23.719234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:23.855872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9fc8-0449-4f7a-8a1a-16d4e052d77c None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:23.857932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:23.903897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:23.904575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:23.904913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:23.905217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:23.912921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:23.914390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:23.929163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:23.931216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:23.932264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 606/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:23.937697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:24.064238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e9b50-8b1c-4c62-9954-9d5dd2c19f2e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:24.109848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:24.110304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:24.110617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:24.110883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:24.111704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:24.111989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:24.114078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:24.133116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 607/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:24.139268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:24.259977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222cba-1d60-47ae-8cc1-f47c1fa1b788 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:24.304230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:24.305013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:24.305489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:24.305887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:24.341257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:24.388949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:24.390431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:24.522287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 607/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:40:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:40:24.524188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:24.545983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:24.641617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcdc20-753e-493b-b877-817cc4d0f6b4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:24.679408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:24.680143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:24.680441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:24.680714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:24.692290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:24.693836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:24.710564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:24.712508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:24.714480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:24.714480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:24.714480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:24.714480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:24.714480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 608/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:24.722157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:24.813448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:24.866552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fdc99-a7fa-4b4d-85ae-9072e0267fd8 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:24.905385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:24.906249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:24.906623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:24.906969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:24.907903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:24.908214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:24.909707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:24.929460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 608/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:24.931712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:25.015660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:25.055577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09495517-9838-4411-8c3c-61462eeeb050 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:25.095429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:25.096115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:25.096444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:25.096741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:25.105622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:25.107311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:25.120752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:25.121932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:25.122926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 609/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:25.127950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:25.239364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4516e-bb5a-44ec-8f8b-992bde14d34d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:25.262477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:25.276892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:25.277585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:25.277962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:25.278297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:25.279106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:25.279477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:25.280956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b44300-b776-4f33-adba-c20c984e04ba None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:25.298584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 609/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:25.300014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:25.422513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7ca0a-befe-4aea-a39e-600a879303b9 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:25.456535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:25.462261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:25.463199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:25.463848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:25.464572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:25.465597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b74de7e13793482882a0097940e339c9) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:25.466425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:25.468513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357abb2-44f8-438f-b490-d5374eea14c2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:25.486470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 610/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:40:25 2022] GET /identity/v3/role_assignments?user.id=b74de7e13793482882a0097940e339c9&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:27.499446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316f34-1439-4acb-ae6d-d8255c8d6e7c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:27.499943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316f34-1439-4acb-ae6d-d8255c8d6e7c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:27.500060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316f34-1439-4acb-ae6d-d8255c8d6e7c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:27.501678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 610/1220] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:27.509850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:27.509850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:27.509850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:27.527013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:27.528252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:27.605425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 611/1221] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:27.630413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf316f34-1439-4acb-ae6d-d8255c8d6e7c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:27.739704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf316f34-1439-4acb-ae6d-d8255c8d6e7c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:27.775953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:27.776558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:27.776846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:27.777164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:27.789498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:27.794942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:27.795981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:27.796837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 611/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:27.802289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:27.914247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d996bd8-41b7-45a8-9bb5-15e1f8cf3df3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:27.938059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:27.950049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:27.950671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:27.950955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:27.951229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:27.951949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] RBAC: Adding query filter params (name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:27.952256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:27.953644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:27.962161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 612/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:40:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:27.969060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.019130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:28.082148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e8de-ce68-4fa6-b390-aa253645c559 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.128214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:28.129384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:28.129667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:28.129976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:28.140131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:28.141586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:28.157488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:28.158672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:28.160349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:28.160349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:28.160349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:28.160349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:28.160349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 612/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:40:28.165140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.273756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edad2f-70ca-40b1-a314-4c52517b813f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.309491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:28.310254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:28.310592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:28.310863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:28.311702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:28.312044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:28.313526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:28.326969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 613/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:28.333337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.433743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:28.441940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e925-9df0-4369-ac3d-4984b76971fa None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.480075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:28.480717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:28.480999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:28.481274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:28.489074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:28.490543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:28.502246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:28.504173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:28.505287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 613/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:28.509574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.560323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:28.615744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af49d-b597-44e0-b7b7-3598b046c7a4 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.651836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:28.652461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:28.652745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:28.653021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:28.653832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:28.654177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:28.655957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:28.679969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 614/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:28.686743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.804431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18e99e-165e-4a7c-9cd9-d86a61e496a0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.845862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:28.846494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:28.846770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:28.847090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:28.859695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:28.861257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:28.877050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:28.878069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:28.879758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:28.879758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:28.879758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:28.879758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:28.879758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 614/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:28.884665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:28.995990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3b8f-d0a0-41fb-bc0f-6a977d9701d7 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:28.998399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:29.032366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:29.033011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:29.033309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:29.033580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:29.034353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:29.035179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:29.036667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:29.048710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 615/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:29.055712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:29.065804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:29.163779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00f1fe-728a-4b40-897e-f13f5cc45428 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:29.204967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:29.205608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:29.205924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:29.206200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:29.213000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:29.214476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:29.226296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:29.227474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:29.228509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 615/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:29.233609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:29.344773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365a101-b472-4c21-856c-e7c422201d41 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:29.382060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:29.382772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:29.383057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:29.383363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:29.384091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:29.384401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:29.385919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2653b1a-fc9c-4a28-ae8c-e0ab36e93174 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:29.401851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 616/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:40:29.404865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:29.473889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:29.516474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d23c0e-3ef4-455e-8ebc-99468444c7e2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:29.551706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:29.552368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:29.552644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:29.552913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b74de7e13793482882a0097940e339c9/roles/75a08ef50cfc402da44f2a28b1702f93` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:29.570772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=b74de7e13793482882a0097940e339c9, role_id=75a08ef50cfc402da44f2a28b1702f93)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:29.572646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ed322-565b-4706-a051-d6e40c82a56f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:29.586198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 616/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:40:29 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b74de7e13793482882a0097940e339c9/roles/75a08ef50cfc402da44f2a28b1702f93 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:40:31.627595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03068fe0-8868-4d2f-8419-cf447504ba48 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:31.628166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03068fe0-8868-4d2f-8419-cf447504ba48 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:31.628166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03068fe0-8868-4d2f-8419-cf447504ba48 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:31.630109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 617/1233] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:31.635401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:31.635810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:31.635956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:31.653500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:31.654690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:31.724693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 617/1234] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:31.748548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03068fe0-8868-4d2f-8419-cf447504ba48 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:31.774502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:31.863308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03068fe0-8868-4d2f-8419-cf447504ba48 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:31.898491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:31.899052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:31.899331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:31.899602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:31.909291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:31.914895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:31.916414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:31.917551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 618/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 29 12:40:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:31.923050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:31.976175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:32.033347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc45a-495b-43dc-8c40-b2075aa4fc10 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.069919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.070589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.070895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.071174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:32.071939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] RBAC: Adding query filter params (name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:32.072271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:32.073782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:32.083792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 618/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 29 12:40:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:32.089781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:32.184309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:32.201395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c6c2-c6a1-442b-bc80-b21ade1cc136 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.238404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.239152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.239487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.239780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:32.252450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:32.254602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:32.271470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:32.272565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:32.273552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:32.273552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:32.273552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:32.273552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:32.273552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 619/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:32.278966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:32.391563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde764-ac6c-42b7-8f65-e3b4e6847d03 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.401494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:32.431348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.431978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.432267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.432534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:32.433344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:32.433719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:32.435508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:32.452062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 619/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:32.457963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:32.571490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687f56a-635a-4361-82d5-24f4c7216764 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.606291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.606939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.607236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.607499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:32.610320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:32.617429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:32.618932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:32.633087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:32.634893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:32.635916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 620/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:32.642474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:32.752598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57e6d-89ca-46f4-8a37-23f586e193db None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.789439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.790092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.790403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.790680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:32.791477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:32.791790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:32.793213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:32.811158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 620/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:40:32.817146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:32.933002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337b8cf-23b5-4d01-a387-d86c8a97fa65 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:32.970004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:32.970627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:32.970922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:32.971202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:33.042300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:33.049595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:33.051894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:33.166233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 621/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:40:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:40:33.171088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:33.201108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:33.282723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c3b35-b397-48cc-a906-ce67900979d1 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:33.327906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:33.328627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:33.328942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:33.329260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:33.340592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:33.342566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:33.359180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:33.360276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:33.361538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:33.361538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:33.361538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:33.361538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:33.361538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 621/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:33.367340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:33.475469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:33.481575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd114-a8da-4707-8e66-e17c70e3251e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:33.518174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:33.518807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:33.519669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:33.519943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:33.520631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:33.520928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:33.522417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:33.535975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 622/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:33.542225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:33.628041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:33.654096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a6d53-cb6d-4f63-9bee-91ccd3076a3f None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:33.692720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:33.693420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:33.693744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:33.694121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:33.701511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:33.703142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:33.715565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:33.716657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:33.717899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 622/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:33.724031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:33.839512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1199e0-ee77-4775-b460-e37d257d2bcf None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:33.878792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:33.879411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:33.879687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:33.880002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:33.880774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:33.881090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:33.882665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705afad2-23fe-4439-8f6f-4e8a92c76040 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:33.896052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 623/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:33.903025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:34.025166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae3e6d-9965-4a16-8e21-812dd2148a19 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:34.061362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:34.062084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:34.062366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:34.062644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:34.063598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] RBAC: Adding query filter params (role.id=75a08ef50cfc402da44f2a28b1702f93, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b74de7e13793482882a0097940e339c9) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:34.063920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:34.065605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09238-2a29-4dfc-b361-dd3588188220 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:34.072913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 623/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:40:33 2022] GET /identity/v3/role_assignments?user.id=b74de7e13793482882a0097940e339c9&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=75a08ef50cfc402da44f2a28b1702f93 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:36.290735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ae42-f423-43bd-b1ed-cb25a98b57a2 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:36.291322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ae42-f423-43bd-b1ed-cb25a98b57a2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:36.291911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ae42-f423-43bd-b1ed-cb25a98b57a2 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:36.293280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 624/1247] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:36.298813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:36.299281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:36.299477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:36.306629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:36.321017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:36.322280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:36.403419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 624/1248] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:36.428800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005ae42-f423-43bd-b1ed-cb25a98b57a2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:36.451341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:36.546796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005ae42-f423-43bd-b1ed-cb25a98b57a2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:36.584702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:36.585360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:36.585632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:36.585934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:36.597950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:36.602948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:36.604241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:36.605086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 625/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:36.610431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:36.724670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f535c2-3586-473b-9694-897c7aa986f7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:36.764093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:36.771101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:36.771871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:36.772087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:36.772385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:36.773178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:36.773495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:36.775018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:36.785121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 625/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:36.791294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:36.887509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:36.914450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957e28-a833-4fbe-ac3d-00b2c9f30302 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:36.953194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:36.953942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:36.954240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:36.954543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:36.965372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:36.966977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:36.984629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:36.985748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:36.986905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:36.986905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:36.986905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:36.986905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:36.986905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 626/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:36.992463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:37.106078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a04b9-5657-4368-ab11-eb5556182574 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:37.145398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:37.146149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:37.146435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:37.146711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:37.147519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:37.147856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:37.149398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:37.163353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 626/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:37.169793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:37.287735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bdba0-b4a2-4455-9174-3c574568415e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:37.317547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:37.326318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:37.326982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:37.327258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:37.327545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:37.336387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:37.338331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:37.351701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:37.353051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:37.354239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 627/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:40:37.361182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:37.425663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:37.477725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7407b4-795e-4a8d-8a04-039b9fe6847d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:37.519323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:37.519951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:37.520239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:37.520517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:37.521410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:37.521783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:37.523456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:37.542558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 627/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:37.548417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:37.669114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a057d-3887-46f5-a36c-06a435443051 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:37.707066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:37.707722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:37.708005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:37.708282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:37.770611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:37.784157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:37.786393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:37.909076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 628/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:40:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:40:37.916903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:38.031562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9802-24b5-4336-9d20-b10ef91317b0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:38.045698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:38.067466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:38.068136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:38.068451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:38.068755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:38.079730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:38.081705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:38.098254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:38.099438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:38.105149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:38.105149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:38.105149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:38.105149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:38.105149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 628/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:38.106642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:38.215084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:38.229906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4869-16c2-4451-9089-9ea01dac222d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:38.267772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:38.268423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:38.268744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:38.269010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:38.269701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:38.270032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:38.271402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:38.282793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 629/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:38.289112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:38.409395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5a9cc-a912-45a1-bca1-a2190c0db3d0 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:38.446319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:38.446961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:38.447245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:38.447525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:38.454434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:38.455865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:38.467847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:38.468782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:38.475717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 629/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:38.476211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:38.584304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ba15e-52b8-4051-a9b7-8a198e4d749e None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:38.621273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:38.622017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:38.622321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:38.622580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:38.623439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:38.623760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:38.625452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90384-5913-4931-9910-d1cc86ea5a84 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:38.645531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 630/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:38.646967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:38.674906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:38.767248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d0da5-50cf-4373-a671-5ac8e953b37d None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:38.806875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:38.807564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:38.807889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:38.808157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:38.809092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b74de7e13793482882a0097940e339c9) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:38.809498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:38.811282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e34f-af2d-4e4e-9ed7-c2b5467beaf9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:38.818487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 630/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:40:38 2022] GET /identity/v3/role_assignments?user.id=b74de7e13793482882a0097940e339c9&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:40.876196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5232-e26a-4016-929d-28c73a0afc97 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:40.876729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5232-e26a-4016-929d-28c73a0afc97 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:40.876729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5232-e26a-4016-929d-28c73a0afc97 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:40.878198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 631/1261] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:40.883246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:40.883620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:40.883800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:40.901463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:40.902674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:40.968763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 631/1262] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:40.991859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf5232-e26a-4016-929d-28c73a0afc97 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.002558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.091214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf5232-e26a-4016-929d-28c73a0afc97 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.124187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.124824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.125109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.125387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.135224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:41.140033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:41.141123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:41.142195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 632/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:41.147662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.166224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.246134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fc671-b656-4d83-bfa9-8dd155e47d77 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.279375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.279982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.280269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.280548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.281267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:41.281573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:41.282958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:41.291019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 632/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:41.296613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.361948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.395298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a612-0c82-4e6f-8081-5c3cd500f31a None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.428276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.428896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.429179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.429456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.439121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:41.440623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:41.454703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:41.455737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:41.456709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:41.456709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:41.456709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:41.456709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:41.456709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 633/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:41.461472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.541371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.561505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f199-187a-47cf-959c-a865ec314397 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.593217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.593838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.594120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.594392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.595071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:41.595372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:41.597212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:41.608360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 633/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:41.614576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.711549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5dadb-1c91-4895-87a7-d3bdb0943a61 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.730834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.745235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c62895-e775-4065-bbdf-88718264766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.745884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62895-e775-4065-bbdf-88718264766f None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.746165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62895-e775-4065-bbdf-88718264766f None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.746438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62895-e775-4065-bbdf-88718264766f None admin] PATH_INFO: `/v3/users/placement` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.753959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c62895-e775-4065-bbdf-88718264766f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:41.755930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c62895-e775-4065-bbdf-88718264766f None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-39c62895-e775-4065-bbdf-88718264766f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:41.766908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:41.768006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:41.769137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 634/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:41.774079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:41.881352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260884-ff49-45ba-af17-26a4d2f15bf4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:41.917447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:41.918104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:41.918357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:41.918632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:41.919430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:41.919738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:41.921159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:41.930266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:41.939125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 634/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:41.945482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c62895-e775-4065-bbdf-88718264766f None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:42.041007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c62895-e775-4065-bbdf-88718264766f None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:42.073342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:42.073989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:42.074269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:42.074540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:42.085739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:42.087200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:42.101574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:42.102623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:42.103909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:42.103909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:42.103909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:42.103909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:42.103909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 635/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:42.108399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:42.205202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae788968-74ed-4eda-a4dd-5c74465c72d4 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:42.239357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:42.239972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:42.240255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:42.240529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:42.241216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:42.241518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:42.242906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:42.254318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 635/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:42.259984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:42.291618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:42.361921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad070f-03ec-483b-ab39-dbc549fb141d None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:42.395264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:42.395897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:42.396180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:42.396455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:42.402966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:42.404451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:42.415557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:42.416594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:42.417540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 636/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:42.422717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:42.479916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:42.520685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee4030-7f1c-4a52-80c2-07aa0f8204dc None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:42.553124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:42.553678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:42.553989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:42.554262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:42.554996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:42.555301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:42.556755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330f9f1-8343-4fba-bad4-dac486be0aa6 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:42.569037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 636/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:42.575062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:42.663625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:42.672463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690f883-2d07-4f60-967c-cecbd2e46a09 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:42.711048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:42.711731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:42.712052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:42.712352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] PATH_INFO: `/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b74de7e13793482882a0097940e339c9/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:42.714383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfaace737a9483dade84c96710c7e54, user_id=b74de7e13793482882a0097940e339c9, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:42.715892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed69f-7ead-4e52-a1b9-1f6ea065cf65 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:42.729087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 637/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 29 12:40:42 2022] PUT /identity/v3/projects/2dfaace737a9483dade84c96710c7e54/users/b74de7e13793482882a0097940e339c9/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:40:44.728210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b67449-5013-4495-b6f9-36e9fb35becc None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:44.728764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b67449-5013-4495-b6f9-36e9fb35becc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:44.728764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b67449-5013-4495-b6f9-36e9fb35becc None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:44.730139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 637/1274] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:44.735145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:44.735515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:44.735712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:44.757559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:44.758781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:44.840367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 638/1275] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:44.867236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b67449-5013-4495-b6f9-36e9fb35becc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:44.969307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b67449-5013-4495-b6f9-36e9fb35becc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:44.986701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:45.004294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.004921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.005205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.005481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.015261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 12:40:45.020120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:45.021097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:45.022037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 638/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 29 12:40:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 12:40:45.027257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.110532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:45.130159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050564-71c6-414e-982d-52ce5119bba5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.162304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.162915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.163194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.163467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.164172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:45.164476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:45.165856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:45.173540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 639/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:45.179801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.279449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a3e3-5368-4180-9334-3f39d34093a6 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.311096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.311712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.311993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.312267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.322932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:45.324374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:45.338203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:45.339227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:45.340323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:45.340323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:45.340323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:45.340323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:45.340323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 639/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:45.347124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.443496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b390fb6-a2e3-40eb-8cd9-de57c320dbd8 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.475609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.476221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.476499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.476771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.477452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:45.477755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:45.479124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:45.491444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 640/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:45.497052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.526984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:45.595987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3d41-b1ce-4de0-b21b-e851e191b9ee None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.628437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.629059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.629342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.629620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.636782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:45.638236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 12:40:45.649744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:45.650865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:45.652036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 640/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:45.656869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.661283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:45.754689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa11b-b9e5-4a15-9a13-cec322ecf0a5 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.786658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.787264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.787544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.787819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:45.788601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:45.788906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:45.790353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:45.815856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 641/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:45.822985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:45.912370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:45.921831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8548b-a248-4569-9a87-868747caa109 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:45.954618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:45.955214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:45.955491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:45.955761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.020293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.021729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.106591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:46.118066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 641/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 29 12:40:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 12:40:46.123710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:46.183579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:46.220842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b703-e9fa-4d82-8984-dddd1b29e8af None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:46.254252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:46.254883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:46.255164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:46.255440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.265125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.266588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 12:40:46.280866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:46.281982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:46.283079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 12:40:46.283079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 12:40:46.283079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 12:40:46.283079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:46.283079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 642/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:46.290732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:46.386200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e4b36-e9ba-4f29-b7b5-2f82c14494a2 None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:46.421231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:46.421855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:46.422137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:46.422404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.423085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:46.423382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.424750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.435721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 642/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 29 12:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:46.441322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:46.538602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7d88d-1ae0-454a-b22f-6ba5a7f7bcbf None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:46.551640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:46.571221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:46.571857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:46.572142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:46.572419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.578862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.580292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 12:40:46.591882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:46.593170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: ERROR keystone.server.flask.application Jul 29 12:40:46.594441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 643/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 29 12:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:46.599185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:46.648906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:46.699212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4ba58-08ab-44b3-949d-575db0d3d09e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:46.732134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:46.732753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:46.733037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:46.733315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.734142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:46.734540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.736611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362cfd3-9c89-4c0d-8db7-e6959eb27ff9 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.754961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 643/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 29 12:40:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:46.762606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:46.865726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8188-64a2-481f-83cc-05b132c748e0 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:46.898206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:46.898805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:46.899087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:46.899359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:46.900137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] RBAC: Adding query filter params (role.id=9419bac9c2624887a6ae8f158b33c48d, scope.project.id=2dfaace737a9483dade84c96710c7e54, user.id=b74de7e13793482882a0097940e339c9) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:46.900442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:46.901874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583293b-6aa8-4105-92b1-aa843265eecc None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:46.907498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 644/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 29 12:40:46 2022] GET /identity/v3/role_assignments?user.id=b74de7e13793482882a0097940e339c9&scope.project.id=2dfaace737a9483dade84c96710c7e54&role.id=9419bac9c2624887a6ae8f158b33c48d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:48.897562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55c1df-c5f6-4705-b7df-bd3a4cb5166c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:48.898135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55c1df-c5f6-4705-b7df-bd3a4cb5166c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:48.898135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55c1df-c5f6-4705-b7df-bd3a4cb5166c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:48.899485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 644/1288] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:48.904487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:48.904919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:48.905076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:48.925210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:48.926717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:48.955487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:48.991755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 645/1289] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:49.013730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55c1df-c5f6-4705-b7df-bd3a4cb5166c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:49.111201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55c1df-c5f6-4705-b7df-bd3a4cb5166c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:49.122186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:49.146086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:49.146711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:49.146992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:49.147266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] PATH_INFO: `/v3/services/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:49.147936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:49.149238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:49.157858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:49.159173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:49.160243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:49.161985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 645/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:40:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:49.164941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:49.260624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33a045-ae1c-44ea-a847-e6a26098da61 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:49.292974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:49.293577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:49.293884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:49.294164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:49.294835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:49.295141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:49.296488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce39079-95dc-4308-a277-5a6d32e885d1 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:49.301659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 646/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 29 12:40:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:49.307249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:49.403521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4191-73ff-45cc-984b-d7ffc2b9c5ea None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:49.435640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:49.436228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:49.436514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:49.436786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] PATH_INFO: `/v3/services` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:49.437448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] RBAC: Adding query filter params (type=placement) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:49.437746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:49.439124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a14451-e3f3-4e94-8ae2-5145fc0974ea None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:49.443403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 646/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 29 12:40:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:40:51.450450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9851f7-b651-4317-85b9-69d0f086957b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:51.451104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9851f7-b651-4317-85b9-69d0f086957b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:51.451104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9851f7-b651-4317-85b9-69d0f086957b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:51.452285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 647/1293] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:51.457791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f0b0-a16a-4cf3-ac3a-1bf8009b1fb5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:51.458250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f0b0-a16a-4cf3-ac3a-1bf8009b1fb5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:51.458383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f0b0-a16a-4cf3-ac3a-1bf8009b1fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:51.476357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d7e3f0b0-a16a-4cf3-ac3a-1bf8009b1fb5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:51.477544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d7e3f0b0-a16a-4cf3-ac3a-1bf8009b1fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:51.507377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:51.545220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 647/1294] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 12:40:51.568025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9851f7-b651-4317-85b9-69d0f086957b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:51.599524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:51.670476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9851f7-b651-4317-85b9-69d0f086957b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:51.703695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:51.704247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:51.704548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:51.704812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:51.705510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:51.707432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da456-2ba0-497b-a34d-12285ebf2230 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:51.722137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 648/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 29 12:40:51 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:40:53.721418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26b1e-d4a8-4231-8e26-89d5daa44525 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:53.721999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26b1e-d4a8-4231-8e26-89d5daa44525 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:53.721999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26b1e-d4a8-4231-8e26-89d5daa44525 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:53.723353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 648/1296] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:53.728450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:53.728833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:53.729002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:53.746440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:53.747598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:53.823122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 649/1297] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:53.848053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c26b1e-d4a8-4231-8e26-89d5daa44525 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:53.953732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c26b1e-d4a8-4231-8e26-89d5daa44525 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:53.988180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:53.988750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:53.989029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:53.989299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] PATH_INFO: `/v3/services/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:53.989943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:53.991298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:53.999835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:54.001185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:54.002188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 649/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:40:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:54.007771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:54.025646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:54.105659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6180-eaee-45aa-b7c2-b94385b2375f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:54.139386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:54.140028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:54.140308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:54.140581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:54.141247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:54.141548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:54.142938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:54.148443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 650/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 29 12:40:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:54.154846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:54.185536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:54.251691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5981c6f-15b4-4145-a04d-cc6578a7886e None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:54.282974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:54.284094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:54.284377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:54.284648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:54.285381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] RBAC: Adding query filter params (interface=public, service_id=3f95fb6228ba4e22a74da7f591bc8c00, region_id=RegionOne) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:54.285684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:54.287056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395018b6-a4ab-4902-bf04-5cfc3d8fc1b2 None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:54.291868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 650/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 29 12:40:54 2022] GET /identity/v3/endpoints?service_id=3f95fb6228ba4e22a74da7f591bc8c00&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:40:54.297489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:54.376444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:54.395648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca9484-a48c-448f-909b-6e093caadd81 None admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:54.428040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:54.428645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:54.428927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:54.429199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:54.429861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:54.431223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4559c-1f69-420a-8570-57f5cb6b0da2 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:54.437843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 651/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 29 12:40:54 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:40:56.443925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfbeb-d554-46b6-a40d-5d7fcdbb7c44 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:56.444442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfbeb-d554-46b6-a40d-5d7fcdbb7c44 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:56.444442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfbeb-d554-46b6-a40d-5d7fcdbb7c44 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:56.445755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 651/1302] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:40:56.450950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:56.451329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:56.451511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:56.470091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:40:56.471784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:40:56.539711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 652/1303] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:40:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 12:40:56.562967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cfbeb-d554-46b6-a40d-5d7fcdbb7c44 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:56.655453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:56.662843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cfbeb-d554-46b6-a40d-5d7fcdbb7c44 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:56.695691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:56.696261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:56.696542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:56.696817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] PATH_INFO: `/v3/services/placement` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:56.697436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:56.698820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 12:40:56.707425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 12:40:56.708396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application with Lock( Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self._enter() Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application return self.creator() Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application created_value = creator( Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: ERROR keystone.server.flask.application Jul 29 12:40:56.709675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 652/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 29 12:40:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 12:40:56.714544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:56.788771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:40:56.813990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c33e6-0051-4514-b363-ee6c9aecd5a1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:56.849146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:56.849759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:56.850071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:56.850351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] PATH_INFO: `/v3/services` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:56.851033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:40:56.851340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:56.852703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e878b-e603-4387-bfcf-2141ae4652d7 None admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:56.857760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 653/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 29 12:40:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:40:56.863508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:40:56.962065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b113c8-26ef-4af5-b329-8b438402a1cc None admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:40:56.995260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:40:56.995866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:40:56.996150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:40:56.996427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:40:56.997139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:40:56.998528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd03efe-8dcc-4247-9b85-62f272f66dbc None admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:40:57.018157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 653/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:40:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:41:06.854782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=30 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:41:07.014306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:00.289854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33415bb-0d07-42f8-a7ee-cb93e5b8a0a9 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:00.290427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33415bb-0d07-42f8-a7ee-cb93e5b8a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:00.290427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33415bb-0d07-42f8-a7ee-cb93e5b8a0a9 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:00.295750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 654/1307] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:00.297308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961d309-c64a-4497-a4f7-682e89923ac9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:00.298131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961d309-c64a-4497-a4f7-682e89923ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:00.298611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961d309-c64a-4497-a4f7-682e89923ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:00.379672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1961d309-c64a-4497-a4f7-682e89923ac9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:00.381194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1961d309-c64a-4497-a4f7-682e89923ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:00.615909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 654/1308] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:42:00.673004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:00.673320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:00.673609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:00.675112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 655/1309] 10.222.0.66 () {54 vars in 965 bytes} [Fri Jul 29 12:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:00.681326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b7868-5b84-4b78-9814-367522666439 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:00.681624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b7868-5b84-4b78-9814-367522666439 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:00.681930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b7868-5b84-4b78-9814-367522666439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:00.775555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-397b7868-5b84-4b78-9814-367522666439 None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:00.776765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-397b7868-5b84-4b78-9814-367522666439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:00.800028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:00.863674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 655/1310] 10.222.0.66 () {58 vars in 1062 bytes} [Fri Jul 29 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:00.872779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:00.874549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:00.996837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197447e-5395-467f-b0e4-f7595ebe4f27 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:01.085459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:01.086103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:01.086514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:01.086798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:01.088509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:01.155020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:01.221778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:01.223497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dbcef-52a4-4787-beb3-335ce71e1812 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:01.244429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 656/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:03.562902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:03.563416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:03.563541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:03.564970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 656/1312] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:03.570121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849ee4-5954-4223-8eff-5d55afa40e1a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:03.570543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849ee4-5954-4223-8eff-5d55afa40e1a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:03.570738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33849ee4-5954-4223-8eff-5d55afa40e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:03.661284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-33849ee4-5954-4223-8eff-5d55afa40e1a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:03.662581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-33849ee4-5954-4223-8eff-5d55afa40e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:03.788134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 657/1313] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:03.828384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:03.830101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:03.945779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d2196-25cf-4088-adda-a9de24e4408c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:04.024816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:04.027771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:04.028441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:04.028849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:04.029126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:04.031420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:04.154651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:04.156723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12daf52-6f50-4a0a-99be-c1684c7e07fd service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:04.176626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 657/1314] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:06.510037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10809e62-b47c-40b5-911f-ca0146522d7a None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:06.510692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10809e62-b47c-40b5-911f-ca0146522d7a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:06.510692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10809e62-b47c-40b5-911f-ca0146522d7a None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:06.511920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 658/1315] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:06.517047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3388a2-ca53-42fd-ac95-1cdc594fc79b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:06.517482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3388a2-ca53-42fd-ac95-1cdc594fc79b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:06.517579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3388a2-ca53-42fd-ac95-1cdc594fc79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:06.588851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-fb3388a2-ca53-42fd-ac95-1cdc594fc79b None None] MFA Rules not processed for user `4c88a9c9720f4b08add7dd0deda4313d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:06.590109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fb3388a2-ca53-42fd-ac95-1cdc594fc79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:06.642320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:06.687231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 658/1316] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:06.742051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:06.742344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:06.742627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:06.744025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 659/1317] 10.222.0.66 () {54 vars in 965 bytes} [Fri Jul 29 12:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:06.749761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a3c38-2ee2-446b-823a-f498c7c13890 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:06.750097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a3c38-2ee2-446b-823a-f498c7c13890 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:06.750376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a3c38-2ee2-446b-823a-f498c7c13890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:06.826427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e08a3c38-2ee2-446b-823a-f498c7c13890 None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:06.827634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e08a3c38-2ee2-446b-823a-f498c7c13890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:06.903911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 659/1318] 10.222.0.66 () {58 vars in 1062 bytes} [Fri Jul 29 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:06.913064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:06.914671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:06.924882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:07.016831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208528e5-5d52-49e6-b4bf-ab2ce5120758 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:07.089364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:07.089940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:07.090226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:07.090504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:07.092141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:07.157628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:07.203575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:07.205026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b486f8-6774-4b32-b493-b1e679728d9b service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:07.223865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 660/1319] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:10.199543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:10.200033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:10.200180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:10.201439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 660/1320] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:10.207238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48828-4118-4ad4-9e08-19734060fae0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:10.207655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48828-4118-4ad4-9e08-19734060fae0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:10.207798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48828-4118-4ad4-9e08-19734060fae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:10.273526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-63b48828-4118-4ad4-9e08-19734060fae0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:10.274753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-63b48828-4118-4ad4-9e08-19734060fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:10.368331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 661/1321] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:10.408976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:10.410556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:10.460057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:10.514008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434778ba-e9c9-4317-918a-7d55bc17a57d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:10.589363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:10.590037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:10.590350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:10.590623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:10.592664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:10.691923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:10.706902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:10.708556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b8549-0db6-4386-8fda-6ad17662d7ba service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:10.728404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 661/1322] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:13.408674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:13.409180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:13.409180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:13.410533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 662/1323] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:13.415898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2a03-3c1c-4250-864e-2e2fd0010aab None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:13.416351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2a03-3c1c-4250-864e-2e2fd0010aab None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:13.416459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba2a03-3c1c-4250-864e-2e2fd0010aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:13.483017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f1ba2a03-3c1c-4250-864e-2e2fd0010aab None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:13.484739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f1ba2a03-3c1c-4250-864e-2e2fd0010aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:13.582798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 662/1324] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:13.621220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:13.622795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:13.643791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:13.734677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ef279-e423-42e8-9f6f-206929b52f75 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:13.810965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:13.811621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:13.811966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:13.812273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:13.814165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:13.885365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:13.923371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:13.924816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530bbe2-4132-4c3e-a5b6-59ef06284293 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:13.942503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 663/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:16.019227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:16.019820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:16.019820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:16.021143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 663/1326] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:16.028035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf3f4c-20fd-40cd-b2f8-fe02d388c4ed None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:16.028469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf3f4c-20fd-40cd-b2f8-fe02d388c4ed None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:16.028770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf3f4c-20fd-40cd-b2f8-fe02d388c4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:16.124148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2acf3f4c-20fd-40cd-b2f8-fe02d388c4ed None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:16.125372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2acf3f4c-20fd-40cd-b2f8-fe02d388c4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:16.225363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 664/1327] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:16.262334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:16.263889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:16.340068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:16.384616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2170545-a7d7-445b-831d-aa807a1aa6ac None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:16.464099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:16.464715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:16.464995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:16.465265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:16.466965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:16.582649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:16.585359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:16.587285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570f171-c673-4b64-9f4d-5b6104fc4bcc service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:16.606158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 664/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:18.660968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:18.661565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:18.661565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:18.662990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 665/1329] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:18.668794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a2ec6-7cd7-44c6-b1e8-aedba378c01d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:18.669370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a2ec6-7cd7-44c6-b1e8-aedba378c01d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:18.669370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a2ec6-7cd7-44c6-b1e8-aedba378c01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:18.736253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6c9a2ec6-7cd7-44c6-b1e8-aedba378c01d None None] MFA Rules not processed for user `4c88a9c9720f4b08add7dd0deda4313d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:18.737457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6c9a2ec6-7cd7-44c6-b1e8-aedba378c01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:18.836159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 665/1330] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:18.873969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:18.875623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:18.907189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:18.986165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1abef6-bf8d-485a-b55e-1bd34dbc66d5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:19.059213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:19.059834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:19.060117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:19.060391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:19.062545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:19.141969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:19.175552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:19.177008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4606e-0c64-4bfb-a4a7-26bf7de80243 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:19.196165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 666/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:21.524889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:21.525417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:21.525417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:21.526790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 666/1332] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:21.532357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6937-c289-482f-8c33-3c924d19fb9d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:21.532762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6937-c289-482f-8c33-3c924d19fb9d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:21.532903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6937-c289-482f-8c33-3c924d19fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:21.597681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e58d6937-c289-482f-8c33-3c924d19fb9d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:21.598891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e58d6937-c289-482f-8c33-3c924d19fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:21.693450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 667/1333] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:21.730084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:21.731623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:21.802434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:21.838201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca981-055a-4329-a731-2a1eff4595f2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:21.911317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:21.911926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:21.912210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:21.912482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:21.914116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:22.023139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:22.024658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cee40-5429-47a1-9d06-ebe31ee6f53c service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:22.035399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:22.045746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 667/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:24.121078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000178f-c369-469c-91fb-65122553dd09 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:24.121078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000178f-c369-469c-91fb-65122553dd09 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:24.121579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000178f-c369-469c-91fb-65122553dd09 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:24.122736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 668/1335] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:24.128535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a32a2-57eb-4fa4-adf6-62bb464535b7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:24.128995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a32a2-57eb-4fa4-adf6-62bb464535b7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:24.129110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a32a2-57eb-4fa4-adf6-62bb464535b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:24.194413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-580a32a2-57eb-4fa4-adf6-62bb464535b7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:24.195626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-580a32a2-57eb-4fa4-adf6-62bb464535b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:24.292483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 668/1336] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:42:24.333259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000178f-c369-469c-91fb-65122553dd09 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:24.334874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e000178f-c369-469c-91fb-65122553dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:24.376015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:24.439202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000178f-c369-469c-91fb-65122553dd09 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:24.510787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:24.511396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:24.511702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:24.512003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:24.513656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:24.597839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:24.623108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:24.624582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11202276-f435-492d-ba7b-3866e8fa1a35 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:24.642541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 669/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:26.989391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:26.989976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:26.989976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:26.991341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 669/1338] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:26.996444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce437b-518e-4b0e-803b-1ea5f1b9fedf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:26.996865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce437b-518e-4b0e-803b-1ea5f1b9fedf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:26.997032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce437b-518e-4b0e-803b-1ea5f1b9fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:27.062609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4fce437b-518e-4b0e-803b-1ea5f1b9fedf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:27.063811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4fce437b-518e-4b0e-803b-1ea5f1b9fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:27.166138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 670/1339] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:27.202700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:27.204244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:27.284379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:27.313495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76db24f-e55b-4206-bc7c-03a35323a830 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:27.386642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:27.387274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:27.387557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:27.387838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:27.389521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:27.496222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:27.497657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0bb2c-35e7-48ff-8910-acf1c54711c6 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:27.525901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 670/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:31.436962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:31.437507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:31.437507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:31.438892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 671/1341] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:31.444504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970243a-103f-47c5-849b-1529a70cce2d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:31.444941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970243a-103f-47c5-849b-1529a70cce2d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:31.445068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970243a-103f-47c5-849b-1529a70cce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:31.450688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:31.512734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5970243a-103f-47c5-849b-1529a70cce2d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:31.513968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5970243a-103f-47c5-849b-1529a70cce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:31.612085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 671/1342] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:31.654369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:31.656592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:31.705100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:31.761510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06927c03-a6f4-4bb3-b12a-4622722e9ae5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:31.839136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:31.839752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:31.840030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:31.840301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:31.841929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:31.937071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:31.954068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:31.955564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ad27-5a41-4e7f-8976-370e4ae59da8 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:31.976493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 672/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:42:34.643294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc42381-c264-459a-9df2-322140c2038c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:34.643869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc42381-c264-459a-9df2-322140c2038c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:34.643869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc42381-c264-459a-9df2-322140c2038c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:34.645219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 672/1344] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:34.651077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995aa6-99d7-4627-9fb6-7983b55ee0b4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:34.651616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995aa6-99d7-4627-9fb6-7983b55ee0b4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:34.651750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995aa6-99d7-4627-9fb6-7983b55ee0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:34.725309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-15995aa6-99d7-4627-9fb6-7983b55ee0b4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:34.726578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-15995aa6-99d7-4627-9fb6-7983b55ee0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:34.828778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 673/1345] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:34.867044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc42381-c264-459a-9df2-322140c2038c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:34.868585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dfc42381-c264-459a-9df2-322140c2038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:34.954946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:34.973615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc42381-c264-459a-9df2-322140c2038c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:35.048348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:35.048965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:35.049243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:35.049512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:35.051157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:35.161741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:35.163212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d8061-2428-45ec-b70b-71dfa8fb9788 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:35.181286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 673/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:39.214674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:39.215273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:39.215273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:39.216607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 674/1347] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:39.222617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dafa-4b2a-4dc4-8207-b46def7c4a35 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:39.223046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dafa-4b2a-4dc4-8207-b46def7c4a35 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:39.223185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dafa-4b2a-4dc4-8207-b46def7c4a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:39.239228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:39.298108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0692dafa-4b2a-4dc4-8207-b46def7c4a35 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:39.299269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0692dafa-4b2a-4dc4-8207-b46def7c4a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:39.425038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 674/1348] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:39.462800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:39.464349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:39.525707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:39.577041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480aec0-a0cc-4086-b333-4e57443fa847 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:39.653931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:39.654567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:39.655397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:39.655671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:39.657330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:39.770558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:39.774317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:39.776307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3052d04-469e-4729-9bf3-89e64ed9180e service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:39.796352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 675/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:44.728439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28bdbd-5c74-4a05-b357-92022230e6d5 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:44.730022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28bdbd-5c74-4a05-b357-92022230e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:44.730022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28bdbd-5c74-4a05-b357-92022230e6d5 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:44.731031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 675/1350] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:44.750833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc6624-d17d-416b-8991-8004a59ea6c3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:44.751324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc6624-d17d-416b-8991-8004a59ea6c3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:44.751324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc6624-d17d-416b-8991-8004a59ea6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:44.868954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c7dc6624-d17d-416b-8991-8004a59ea6c3 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:44.870187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c7dc6624-d17d-416b-8991-8004a59ea6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:44.988578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 676/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:42:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:49.212941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deff61-41c4-4f03-9c2c-f8c49e7948cf None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:49.213495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deff61-41c4-4f03-9c2c-f8c49e7948cf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:49.213495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deff61-41c4-4f03-9c2c-f8c49e7948cf None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:49.215215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 676/1352] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:49.221164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a250-a9d6-4bc0-bd60-5edddca2dea6 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:49.221636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a250-a9d6-4bc0-bd60-5edddca2dea6 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:49.221732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a250-a9d6-4bc0-bd60-5edddca2dea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:49.276010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:49.300922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-04c7a250-a9d6-4bc0-bd60-5edddca2dea6 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:49.302082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-04c7a250-a9d6-4bc0-bd60-5edddca2dea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:49.423127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=59 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:49.432587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 677/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:42:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:50.673608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e34d9-a5e8-410b-a1d6-83eae4e0a6cd None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:50.674165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e34d9-a5e8-410b-a1d6-83eae4e0a6cd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:50.674378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e34d9-a5e8-410b-a1d6-83eae4e0a6cd None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:50.676039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 677/1354] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:50.681351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268e7d-5bde-449e-949d-936be9ef5a99 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:50.681758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268e7d-5bde-449e-949d-936be9ef5a99 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:50.681935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49268e7d-5bde-449e-949d-936be9ef5a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:50.760749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-49268e7d-5bde-449e-949d-936be9ef5a99 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:50.761984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-49268e7d-5bde-449e-949d-936be9ef5a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:50.806973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:50.876006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 678/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:42:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:54.299281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ba88-0c92-4347-892a-f5b66d0ec0a5 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:54.300151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ba88-0c92-4347-892a-f5b66d0ec0a5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:54.300594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ba88-0c92-4347-892a-f5b66d0ec0a5 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:54.305059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 678/1356] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:54.308009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690494-481e-47e5-aa63-e929d2a26048 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:54.308806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690494-481e-47e5-aa63-e929d2a26048 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:54.309260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c690494-481e-47e5-aa63-e929d2a26048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:54.380708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8c690494-481e-47e5-aa63-e929d2a26048 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:54.382968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8c690494-481e-47e5-aa63-e929d2a26048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:54.485257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 679/1357] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:54.541238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:54.541804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:54.542618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:54.545467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 679/1358] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 29 12:42:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:54.554564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323f22-1b58-4da7-91d5-887e49b7d621 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:54.554845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323f22-1b58-4da7-91d5-887e49b7d621 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:54.555126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323f22-1b58-4da7-91d5-887e49b7d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:54.575371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:54.640682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-94323f22-1b58-4da7-91d5-887e49b7d621 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:54.642422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-94323f22-1b58-4da7-91d5-887e49b7d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:54.769722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 680/1359] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 29 12:42:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:54.777906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:54.779595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:54.933598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b902-f8ad-450a-a243-ca0542ae0333 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:55.010474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:55.065672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:55.066326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:55.066611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:55.066885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:55.068509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:55.166786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:55.168424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb41d1e-661a-494f-88ce-7ce94e72a00c service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:55.187728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 680/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:42:57.430569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:57.430983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:57.431165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:57.432715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 681/1361] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:42:57.438367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a74c-405d-4f4b-aaf6-7a6cc6adcff5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:57.438797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a74c-405d-4f4b-aaf6-7a6cc6adcff5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:57.438977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a74c-405d-4f4b-aaf6-7a6cc6adcff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:57.511883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9cc4a74c-405d-4f4b-aaf6-7a6cc6adcff5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:42:57.513232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9cc4a74c-405d-4f4b-aaf6-7a6cc6adcff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:57.554191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:57.619895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 681/1362] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:42:57.667945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:42:57.669970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:57.751027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:57.857710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653749ad-2f86-4261-9721-a9b5f4168027 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:42:57.994581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:42:57.995201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:42:57.995540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:42:57.995814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:42:57.997523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:42:58.025387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:42:58.104229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:42:58.106239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76217512-8cbe-4729-a905-7559f64ab186 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:42:58.129327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 682/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:43:00.496467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552656b-904e-49b0-8e42-c0da3b39f7f7 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:00.496985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552656b-904e-49b0-8e42-c0da3b39f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:00.496985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552656b-904e-49b0-8e42-c0da3b39f7f7 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:00.499044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 682/1364] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:00.504217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabb386-f9dc-4740-984c-6c8d2a6a7b69 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:00.504681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabb386-f9dc-4740-984c-6c8d2a6a7b69 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:00.504797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabb386-f9dc-4740-984c-6c8d2a6a7b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:00.588231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ceabb386-f9dc-4740-984c-6c8d2a6a7b69 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:00.589478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ceabb386-f9dc-4740-984c-6c8d2a6a7b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:00.695566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:00.704717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 683/1365] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:00.746936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:00.747159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:00.747661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:00.749047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 683/1366] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 29 12:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:00.754984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbd2d-54cc-49a6-b53c-b105c4dc63e9 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:00.755343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbd2d-54cc-49a6-b53c-b105c4dc63e9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:00.755691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbd2d-54cc-49a6-b53c-b105c4dc63e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:00.839194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-32ffbd2d-54cc-49a6-b53c-b105c4dc63e9 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:00.840411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-32ffbd2d-54cc-49a6-b53c-b105c4dc63e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:00.963301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 684/1367] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 29 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:00.974735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:00.976771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:01.154712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb9005-cf2e-4a8d-bc71-78bd2bc4d6e2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:01.176275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:01.277063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:01.277679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:01.277990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:01.278283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:01.279998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:01.376526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:01.378093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dc28b-434d-40ad-95af-c5790582cd6f service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:01.397401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 684/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:02.380252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b070-a2e5-446b-98b5-a5c04db91e55 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:02.380802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b070-a2e5-446b-98b5-a5c04db91e55 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:02.380802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b070-a2e5-446b-98b5-a5c04db91e55 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:02.382371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 685/1369] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:43:02.387244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db84963-ccbf-42b7-85d3-8b502c316ead None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:02.387681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db84963-ccbf-42b7-85d3-8b502c316ead None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:02.387780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db84963-ccbf-42b7-85d3-8b502c316ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:02.444321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:02.483024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8db84963-ccbf-42b7-85d3-8b502c316ead None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:02.484380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8db84963-ccbf-42b7-85d3-8b502c316ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:02.588816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 685/1370] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:02.899060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:02.899353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:02.899631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:02.901121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 686/1371] 10.222.0.66 () {54 vars in 964 bytes} [Fri Jul 29 12:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:02.906296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838828-c998-4e57-9aab-bf71b0e53921 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:02.906688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838828-c998-4e57-9aab-bf71b0e53921 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:02.907198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838828-c998-4e57-9aab-bf71b0e53921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:02.990466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-88838828-c998-4e57-9aab-bf71b0e53921 None None] MFA Rules not processed for user `b1a7571787a4415fb254a652062e50f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:02.991670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-88838828-c998-4e57-9aab-bf71b0e53921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:03.040471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:03.072434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 686/1372] 10.222.0.66 () {58 vars in 1061 bytes} [Fri Jul 29 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:03.080377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:03.082383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:03.138844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:03.196911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2a0cd-193b-4fdc-bfe3-69fb0e9a1c7d None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:03.274903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:03.275527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:03.275804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:03.276071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:03.278020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:03.390920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:03.405194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:03.406779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5ad94-ddfc-45cd-a740-cc234e300b46 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:03.427610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 687/1373] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:03.519553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:03.520043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:03.520155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:03.521473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 687/1374] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:03.526740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e87b3-45bc-4160-adb0-0adcb90bd851 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:03.526959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e87b3-45bc-4160-adb0-0adcb90bd851 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:03.527261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e87b3-45bc-4160-adb0-0adcb90bd851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:03.601322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-904e87b3-45bc-4160-adb0-0adcb90bd851 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:03.602544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-904e87b3-45bc-4160-adb0-0adcb90bd851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:03.718199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 688/1375] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:03.756729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:03.758560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:03.915975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287e996-43a8-4fb9-b295-661530d90052 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:03.968160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:04.029606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:04.030249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:04.030526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:04.030796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:04.032402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:04.127533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:04.129144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cb854-3e45-46c3-abfd-c2dae592895b service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:04.149201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 688/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:06.170243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:06.170704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:06.170800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:06.172150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 689/1377] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:06.177346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605455-8e0b-45e6-98aa-85ad447f706d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:06.177798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605455-8e0b-45e6-98aa-85ad447f706d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:06.178008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5605455-8e0b-45e6-98aa-85ad447f706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:06.244426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b5605455-8e0b-45e6-98aa-85ad447f706d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:06.251560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b5605455-8e0b-45e6-98aa-85ad447f706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:06.259502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:06.363937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 689/1378] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:06.412629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:06.414189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:06.477229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:06.577643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bd806-c712-411e-93fe-d37cfc2fabb7 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:06.690654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:06.691189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:06.691471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:06.691743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:06.693347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:06.711911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:06.785652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:06.787376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd2594-3d5a-4d62-a2fc-057181d1fe05 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:06.805563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 690/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:08.883144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:08.884048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:08.884331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:08.885915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 690/1380] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:08.892028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf58f-b410-4fe2-afcf-08aec543f1da None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:08.892466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf58f-b410-4fe2-afcf-08aec543f1da None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:08.892559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf58f-b410-4fe2-afcf-08aec543f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:08.965635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-16fcf58f-b410-4fe2-afcf-08aec543f1da None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:08.966884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-16fcf58f-b410-4fe2-afcf-08aec543f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:09.076112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:09.085864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 691/1381] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:09.129725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:09.131296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:09.275290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:09.289745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f487a56-10df-47fe-a3bb-2c29df42c259 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:09.417004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:09.417623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:09.417933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:09.418273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:09.420496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:09.517370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:09.519078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64c12-4186-4bf6-846e-fd43b719fa07 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:09.530472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:09.552632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 691/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:11.604407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:11.604967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:11.604967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:11.606378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 692/1383] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:11.612056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7310-d727-4ebb-9265-4e87607ea888 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:11.612552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7310-d727-4ebb-9265-4e87607ea888 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:11.612926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7310-d727-4ebb-9265-4e87607ea888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:11.812987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e01f7310-d727-4ebb-9265-4e87607ea888 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:11.814905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e01f7310-d727-4ebb-9265-4e87607ea888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:11.942725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 692/1384] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:11.987675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:11.989296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:12.151077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db866-4a07-401b-8b5e-c44fdbecd5c1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:12.258445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:12.323927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:12.324929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:12.325502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:12.326522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:12.328744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:12.428074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:12.430141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0c7c-68db-4b7d-8f6a-fd1509e61893 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:12.453649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 693/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:43:14.470802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:14.471353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:14.471353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:14.472715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 693/1386] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:14.478148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd03f4-bba8-44d6-85aa-b364a32e084a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:14.478616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd03f4-bba8-44d6-85aa-b364a32e084a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:14.478865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd03f4-bba8-44d6-85aa-b364a32e084a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:14.547513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-afdd03f4-bba8-44d6-85aa-b364a32e084a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:14.548715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-afdd03f4-bba8-44d6-85aa-b364a32e084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:14.585969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:14.663089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 694/1387] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:43:14.708614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:14.710387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:14.790282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:14.865686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e21aa-41bd-46bb-8dbe-c9b6a704bb4e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:14.986996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:14.987896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:14.988246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:14.988690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:14.990916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:15.042003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:15.101173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:15.103187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcd50f-8379-40a4-8311-412899193299 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:15.125501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 694/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:17.200886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46606-05de-475e-89ba-b918a121d583 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:17.201411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46606-05de-475e-89ba-b918a121d583 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:17.201506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46606-05de-475e-89ba-b918a121d583 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:17.202986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 695/1389] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:17.208174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b296e-465f-438e-80cf-10525418c416 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:17.208644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b296e-465f-438e-80cf-10525418c416 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:17.208741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b296e-465f-438e-80cf-10525418c416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:17.275441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-605b296e-465f-438e-80cf-10525418c416 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:17.276642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-605b296e-465f-438e-80cf-10525418c416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:17.375072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 695/1390] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:17.422720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46606-05de-475e-89ba-b918a121d583 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:17.424259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-64c46606-05de-475e-89ba-b918a121d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:17.571626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46606-05de-475e-89ba-b918a121d583 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:17.588569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:17.689688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:17.690339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:17.690841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:17.691359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:17.693455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:17.783631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:17.785233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8887577-64e0-4a38-af59-7f49edbe456d service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:17.803234 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 696/1391] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:18.513560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d973b9-9f3b-4e40-80f5-74fa4d9dacc9 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:18.514106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d973b9-9f3b-4e40-80f5-74fa4d9dacc9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:18.514106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d973b9-9f3b-4e40-80f5-74fa4d9dacc9 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:18.515417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 696/1392] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:18.520681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b18a-5c22-4020-b337-2e741968ac22 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:18.521116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b18a-5c22-4020-b337-2e741968ac22 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:18.521302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b18a-5c22-4020-b337-2e741968ac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:18.548325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:18.589657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-eea4b18a-5c22-4020-b337-2e741968ac22 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:18.591523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eea4b18a-5c22-4020-b337-2e741968ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:18.688351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 697/1393] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:18.746730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:18.747077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:18.747361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:18.748717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 697/1394] 10.222.0.66 () {54 vars in 966 bytes} [Fri Jul 29 12:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:18.753725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fcfc-7500-4b5e-9a25-5f056741e7e7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:18.754057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fcfc-7500-4b5e-9a25-5f056741e7e7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:18.754335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fcfc-7500-4b5e-9a25-5f056741e7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:18.830549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cc64fcfc-7500-4b5e-9a25-5f056741e7e7 None None] MFA Rules not processed for user `11a9dc108d68404ba926c566516e7812`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:18.831762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cc64fcfc-7500-4b5e-9a25-5f056741e7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:18.872366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:18.907439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 698/1395] 10.222.0.66 () {58 vars in 1063 bytes} [Fri Jul 29 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:18.915088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:18.916642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:18.932386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:19.034587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6898-4e81-493b-8803-53e5d0001349 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:19.118525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:19.119126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:19.119345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:19.119610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:19.121386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:19.189448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:19.231308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:19.232762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71075681-975d-4ae1-98e0-b7fc4f72d6f9 service glance] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:19.251881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 698/1396] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:19.839379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45514d-c693-41e3-aa86-17d0f210d312 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:19.839967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45514d-c693-41e3-aa86-17d0f210d312 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:19.839967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45514d-c693-41e3-aa86-17d0f210d312 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:19.841990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 699/1397] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:43:19.847545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:19.848009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:19.848097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:19.924581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:19.925849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.019656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb009e34-63cc-4253-b9d5-b0838fec441c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:20.020269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb009e34-63cc-4253-b9d5-b0838fec441c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:20.020615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb009e34-63cc-4253-b9d5-b0838fec441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:20.027544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 699/1398] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:20.076758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:20.078008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.092989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-bb009e34-63cc-4253-b9d5-b0838fec441c None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:20.093510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bb009e34-63cc-4253-b9d5-b0838fec441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.104821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:20.190444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 700/1399] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:43:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:43:20.198225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b14713-a761-4aa5-a030-818d8a3dddc2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:20.233571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666a69-62e5-403d-a9ad-24096a211d5f None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:20.233889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666a69-62e5-403d-a9ad-24096a211d5f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:20.234166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46666a69-62e5-403d-a9ad-24096a211d5f None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:20.235532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 701/1400] 10.222.0.66 () {54 vars in 943 bytes} [Fri Jul 29 12:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:43:20.241492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b584-cef5-47ed-8e11-d420ccfb818f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:20.241900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b584-cef5-47ed-8e11-d420ccfb818f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:20.242303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b584-cef5-47ed-8e11-d420ccfb818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:20.312866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:20.315318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:20.316013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:20.316362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:20.316700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:20.318430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.322009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fe84b584-cef5-47ed-8e11-d420ccfb818f None None] MFA Rules not processed for user `a3dffd8d4deb4cadae3402ffc79ad123`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:20.323510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fe84b584-cef5-47ed-8e11-d420ccfb818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.337457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:20.410831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:20.412418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:20.430337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 700/1401] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:43:20.433869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 702/1402] 10.222.0.66 () {58 vars in 1040 bytes} [Fri Jul 29 12:43:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:43:20.441782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:20.443000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.536560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb686-1728-4e24-baa3-c13e558f8329 service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:20.600023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:20.657463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:20.658120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:20.658403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:20.658680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:20.660361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:20.746468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:20.748109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de60137-dc80-4d68-b32f-914033dad1dd service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:20.756226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 701/1403] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:43:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:43:22.501761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:22.502415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:22.502415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:22.503948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 703/1404] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:22.509522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b494b-104d-4b9b-9491-a040c70fa6d1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:22.510555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b494b-104d-4b9b-9491-a040c70fa6d1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:22.510904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b494b-104d-4b9b-9491-a040c70fa6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:22.595150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-119b494b-104d-4b9b-9491-a040c70fa6d1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:22.596402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-119b494b-104d-4b9b-9491-a040c70fa6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:22.641204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:22.706352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 702/1405] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:22.751144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:22.752864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:22.887974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:22.930605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83259ecf-d693-43d2-bfde-9be07d41c2c1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:23.064192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:23.064927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:23.065294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:23.065639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:23.067381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:23.159239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:23.174560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:23.176170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194abb26-1b44-43bc-87ef-ba9d29037916 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:23.195587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 704/1406] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:25.294292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:25.295399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:25.295949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:25.297814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 703/1407] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:43:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:43:25.303435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468505ba-b658-4cce-aa5c-1544b1bb6afc None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:25.303883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468505ba-b658-4cce-aa5c-1544b1bb6afc None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:25.303985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468505ba-b658-4cce-aa5c-1544b1bb6afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:25.379344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-468505ba-b658-4cce-aa5c-1544b1bb6afc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:25.380581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-468505ba-b658-4cce-aa5c-1544b1bb6afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:25.494305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 705/1408] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:43:25.538623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:25.540151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:25.699047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0228b-533c-4ddc-9ea9-5ef402ae47f1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:25.730275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:25.829447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:25.830276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:25.831071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:25.831336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:25.833092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:25.934117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:25.936778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a7002-95ed-412c-a05b-7daef294c4cd service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:25.955475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 704/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:35.491842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=76 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:35.953127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=87 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:42.841583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9231899-bd2e-405e-9729-41d04746f4f6 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:42.842169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9231899-bd2e-405e-9729-41d04746f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:42.842169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9231899-bd2e-405e-9729-41d04746f4f6 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:42.843854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 706/1410] 10.222.0.66 () {54 vars in 918 bytes} [Fri Jul 29 12:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:42.848842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0803ac-b179-41b6-b0e9-eeb3590207f9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:42.849274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0803ac-b179-41b6-b0e9-eeb3590207f9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:42.849398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0803ac-b179-41b6-b0e9-eeb3590207f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:42.925339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1b0803ac-b179-41b6-b0e9-eeb3590207f9 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:42.926592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1b0803ac-b179-41b6-b0e9-eeb3590207f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:43.039834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 705/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Fri Jul 29 12:43:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:43.054847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21d79-b692-478f-a2df-feef0aa32781 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:43.055141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21d79-b692-478f-a2df-feef0aa32781 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:43.055423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa21d79-b692-478f-a2df-feef0aa32781 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:43.056777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 707/1412] 10.222.0.66 () {54 vars in 918 bytes} [Fri Jul 29 12:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:43.061389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb625a-92bd-4d15-9617-c12331f030b5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:43.061676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb625a-92bd-4d15-9617-c12331f030b5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:43.061986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb625a-92bd-4d15-9617-c12331f030b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:43.133805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:43.136463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-36eb625a-92bd-4d15-9617-c12331f030b5 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:43.138317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-36eb625a-92bd-4d15-9617-c12331f030b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:43.254856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 706/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Fri Jul 29 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:45.798172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:45.798790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:45.798790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:45.800087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:45.805305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c75e-4ff3-46fb-bae0-991b5331e8ca None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:45.805741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c75e-4ff3-46fb-bae0-991b5331e8ca None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:45.805929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c75e-4ff3-46fb-bae0-991b5331e8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:45.874762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3609c75e-4ff3-46fb-bae0-991b5331e8ca None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:45.875967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3609c75e-4ff3-46fb-bae0-991b5331e8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:45.933702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:45.977715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:46.021047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:46.022627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:46.172216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661535ff-657e-4f06-a532-6e8ed2fe23a8 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:46.243244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:46.287959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:46.288581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:46.288859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:46.289129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:46.290763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:46.380330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:46.381886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84069bc7-53e1-4a4b-ad4b-080c15918209 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:46.401436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 709/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:47.301764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:47.302360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:47.302360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:47.303716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:47.308644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6adc-82ec-48f5-9fc7-aebe3ec25931 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:47.309603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6adc-82ec-48f5-9fc7-aebe3ec25931 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:47.309786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6adc-82ec-48f5-9fc7-aebe3ec25931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:47.384414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d68e6adc-82ec-48f5-9fc7-aebe3ec25931 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:47.385540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d68e6adc-82ec-48f5-9fc7-aebe3ec25931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:47.412452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:47.483314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:47.630774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:47.632365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:47.776466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ceab5-a664-4fea-b589-518d9da4157f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:47.816347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:47.888481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:47.889093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:47.889366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:47.889633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:47.891264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:47.979594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:47.981718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40541f9-f158-40fe-acc2-7572f7fd7c81 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:47.999692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 709/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:48.893730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:48.894345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:48.894345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:48.895676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:48.900498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b8-11af-4730-a8ce-1eace137dc9c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:48.900945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b8-11af-4730-a8ce-1eace137dc9c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:48.901045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954121b8-11af-4730-a8ce-1eace137dc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:48.955708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:48.978079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-954121b8-11af-4730-a8ce-1eace137dc9c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:48.979283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-954121b8-11af-4730-a8ce-1eace137dc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:49.074386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:49.227784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:49.229213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:49.376048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb877b-e5b4-4805-9491-d4187669abf3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:49.388488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:49.489199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:49.489744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:49.490059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:49.490365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:49.492022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:49.580330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:49.581941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df746b2-fe61-427b-8a61-a7522de475df service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:49.601457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 712/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:50.490644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:50.491187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:50.491187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:50.492445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:50.497773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114aff4-c405-44c0-85d0-642d84b12da7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:50.498219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114aff4-c405-44c0-85d0-642d84b12da7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:50.498365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114aff4-c405-44c0-85d0-642d84b12da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:50.518559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:50.574900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1114aff4-c405-44c0-85d0-642d84b12da7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:50.576105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1114aff4-c405-44c0-85d0-642d84b12da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:50.674797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:50.824928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:50.826470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:50.938106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:50.974537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911243f-933b-4b16-ab22-f89f7453e579 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:51.086028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:51.086564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:51.086842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:51.087112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:51.088716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:51.164974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:51.178575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:51.180154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0307f45-dc05-4abe-8a68-595787fa72c2 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:51.200703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 712/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:52.070270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:52.070821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:52.070821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:52.072187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:52.077843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5465f2-30fc-42fe-b3d4-b49886c39d54 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:52.078268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5465f2-30fc-42fe-b3d4-b49886c39d54 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:52.078427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5465f2-30fc-42fe-b3d4-b49886c39d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:52.155407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9a5465f2-30fc-42fe-b3d4-b49886c39d54 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:52.157024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9a5465f2-30fc-42fe-b3d4-b49886c39d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:52.254439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:52.413039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:52.414520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:52.496635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:52.557517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbdb4a-9337-4457-b348-4156ed8d85b2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:52.689005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:52.689540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:52.689851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:52.690134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:52.691753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:52.739705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:52.781237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:52.782859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8924394-1b7d-424f-b5aa-c3e5f18cbb06 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:52.800498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 715/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:53.621594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:53.622185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:53.622185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:53.623530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:53.628653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aabaa-df10-4957-8cdf-55b084feb33f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:53.629063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aabaa-df10-4957-8cdf-55b084feb33f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:53.629219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aabaa-df10-4957-8cdf-55b084feb33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:53.705608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2e3aabaa-df10-4957-8cdf-55b084feb33f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:53.706859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2e3aabaa-df10-4957-8cdf-55b084feb33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:53.803213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:53.955049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:53.956484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:54.004739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:54.101977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2634d-9279-4595-a9ac-2ee48ef185fd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:54.216832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:54.217380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:54.217966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:54.218238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:54.219845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:54.226720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:54.309437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:54.311050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad86cb-221c-4cc3-8fe9-9cd26bd5df86 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:54.328598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 715/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:55.147421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:55.147961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:55.147961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:55.149350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:55.155104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ea67-393b-46fc-9b36-58c66cb39295 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:55.155541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ea67-393b-46fc-9b36-58c66cb39295 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:55.155678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ea67-393b-46fc-9b36-58c66cb39295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:55.230553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-37f2ea67-393b-46fc-9b36-58c66cb39295 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:55.231748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-37f2ea67-393b-46fc-9b36-58c66cb39295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:55.298511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:55.327706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:55.479353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:55.481400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:55.499439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:55.626557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a84c13-4962-4ead-a709-86cd50372356 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:55.711838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:55.738959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:55.739593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:55.739908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:55.740215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:55.741906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:55.830943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:55.832556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e38536-9cad-4b47-a447-d3e4cf7fb991 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:55.851023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 718/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:56.642370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:56.642910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:56.642910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:56.644256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:56.649050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa521ff-b939-4997-8877-d86a16386b4b None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:56.649492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa521ff-b939-4997-8877-d86a16386b4b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:56.649602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa521ff-b939-4997-8877-d86a16386b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:56.725689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8aa521ff-b939-4997-8877-d86a16386b4b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:56.726907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8aa521ff-b939-4997-8877-d86a16386b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:56.768346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:56.821867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:56.970807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:56.972247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:57.118047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7825e40-500f-4255-8773-982aae9867d1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:57.172687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:57.232281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:57.232836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:57.233118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:57.233394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:57.235058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:57.329029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:57.330685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bfc0-1d72-4b49-b487-b9649a5eaf7a service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:57.348753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 718/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:43:58.152112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:58.152583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:58.152687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:58.154060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:43:58.158963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9676c1-6f18-4531-8a1e-c4e20d97b410 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:58.159353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9676c1-6f18-4531-8a1e-c4e20d97b410 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:58.159523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9676c1-6f18-4531-8a1e-c4e20d97b410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:58.237406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ea9676c1-6f18-4531-8a1e-c4e20d97b410 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:43:58.238654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ea9676c1-6f18-4531-8a1e-c4e20d97b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:58.240683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:58.336317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:43:58.490802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:43:58.492255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:58.639229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7296-b74b-41d2-b97f-584ee20ea7bc None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:43:58.666176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:43:58.753093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:43:58.753629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:43:58.753930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:43:58.754198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:43:58.755788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:43:58.844695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:43:58.846312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdcec3-43e0-4ccf-83c5-d4b93686f47e service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:43:58.864530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 721/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:00.821727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:00.822452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:00.822452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:00.824127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:00.830175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899c76-5834-4025-bc04-ea3df3679429 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:00.830590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899c76-5834-4025-bc04-ea3df3679429 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:00.830719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899c76-5834-4025-bc04-ea3df3679429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:00.872711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:00.897195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-18899c76-5834-4025-bc04-ea3df3679429 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:00.898412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-18899c76-5834-4025-bc04-ea3df3679429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:00.994382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:01.032622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:01.034275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:01.138450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38978d8-098c-4f98-88dc-49f0852fcb8a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:01.170919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:01.212769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:01.213386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:01.213666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:01.213974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:01.215653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:01.327259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:01.328686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c3079-3457-462d-9dba-7b76d15673e0 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:01.347213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 721/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:03.467981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:03.468536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:03.468536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:03.469801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:03.475063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139e80-15b3-498f-bef9-8a3b04c6f8d0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:03.475459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139e80-15b3-498f-bef9-8a3b04c6f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:03.475616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139e80-15b3-498f-bef9-8a3b04c6f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:03.540745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-20139e80-15b3-498f-bef9-8a3b04c6f8d0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:03.541974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-20139e80-15b3-498f-bef9-8a3b04c6f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:03.555085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:03.639004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:03.677211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:03.678862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:03.782957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6698d8-2d8a-40ee-9311-006fcda3c715 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:03.795410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:03.858503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:03.859123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:03.859401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:03.859674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:03.861326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:03.971140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:03.972619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91962-928d-4a6d-b7f4-c623eca78f95 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:03.991068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 724/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:06.286025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:06.286527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:06.286622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:06.288082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:06.293311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e08b-f3dd-48ab-ae87-0b75d52ffbaf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:06.293719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e08b-f3dd-48ab-ae87-0b75d52ffbaf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:06.293901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e08b-f3dd-48ab-ae87-0b75d52ffbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:06.341444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:06.360124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6050e08b-f3dd-48ab-ae87-0b75d52ffbaf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:06.361313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6050e08b-f3dd-48ab-ae87-0b75d52ffbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:06.457258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:06.496454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:06.498025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:06.604397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34ca3a-69fd-455c-8396-7d20373d606e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:06.643893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:06.677917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:06.678532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:06.678816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:06.679090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:06.680716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:06.790677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:06.792215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f430887-c1c5-4a79-85b9-0ef13ea5e08c service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:06.814601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 724/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:08.974825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:08.975390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:08.975390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:08.976761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:08.982535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7709-ef82-42d8-84fa-8e038d46ab51 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:08.982945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7709-ef82-42d8-84fa-8e038d46ab51 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:08.983082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7709-ef82-42d8-84fa-8e038d46ab51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:09.048087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-258e7709-ef82-42d8-84fa-8e038d46ab51 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:09.049293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-258e7709-ef82-42d8-84fa-8e038d46ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:09.070821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:09.149758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:09.189536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:09.191969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:09.295739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623dda-c81f-4e28-b562-89d23acdc26e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:09.315836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:09.369301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:09.369941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:09.370220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:09.370492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:09.372682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:09.478399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:09.479828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd49396-25c5-41ff-931e-f7785c945754 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:09.497864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 727/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:11.522633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:11.523176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:11.523176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:11.524466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:11.530308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce391-2f88-4800-bd3b-e524c0082e71 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:11.530709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce391-2f88-4800-bd3b-e524c0082e71 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:11.530898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce391-2f88-4800-bd3b-e524c0082e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:11.594325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:11.597301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c87ce391-2f88-4800-bd3b-e524c0082e71 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:11.598846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c87ce391-2f88-4800-bd3b-e524c0082e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:11.697727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:11.734456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:11.735997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:11.839944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3f070-f574-4773-b10e-bfbc7b6d6299 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:11.888552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:11.912975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:11.913595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:11.913908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:11.914188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:11.915813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:12.024712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:12.026195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bdc3a-ea45-43a0-95ad-4ff33dd06f97 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:12.045835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 727/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:14.211040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:14.211512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:14.211644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:14.213027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:14.219393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f61b-db42-4bd8-9efe-2b5467f4c4f5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:14.219811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f61b-db42-4bd8-9efe-2b5467f4c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:14.219951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f61b-db42-4bd8-9efe-2b5467f4c4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:14.290684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c2d1f61b-db42-4bd8-9efe-2b5467f4c4f5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:14.291889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c2d1f61b-db42-4bd8-9efe-2b5467f4c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:14.321501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:14.389495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:14.427902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:14.429424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:14.532037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cec846-071d-4a12-8a81-5fdf8cd5117f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:14.560128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:14.604561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:14.605758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:14.606071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:14.606348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:14.607974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:14.716474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:14.718200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce5adf-b488-40b4-9db9-6993a99a5f17 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:14.735921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 730/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:44:16.866067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:16.866618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:16.866618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:16.868011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:16.873772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf632-08df-4d01-8c1f-1b38a81537ab None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:16.874225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf632-08df-4d01-8c1f-1b38a81537ab None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:16.874361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf632-08df-4d01-8c1f-1b38a81537ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:16.940140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-41baf632-08df-4d01-8c1f-1b38a81537ab None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:16.941370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-41baf632-08df-4d01-8c1f-1b38a81537ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:16.950366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:17.038626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:44:17.077278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:17.078944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:17.190879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23103a3-051c-4f86-9936-cbbf36a9d897 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:17.249593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:17.268225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:17.268843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:17.269122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:17.269393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:17.271052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:17.403377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:17.404891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2fdfb-855b-44d4-b9a8-a80b04bffc87 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:17.424759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 730/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:19.509764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:19.510328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:19.510328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:19.512168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:19.517530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9441cc-24fb-4ae6-82a8-b986f1b90db1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:19.517970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9441cc-24fb-4ae6-82a8-b986f1b90db1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:19.518114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9441cc-24fb-4ae6-82a8-b986f1b90db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:19.584754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ec9441cc-24fb-4ae6-82a8-b986f1b90db1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:19.586007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ec9441cc-24fb-4ae6-82a8-b986f1b90db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:19.625962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:19.685402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:19.724430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:19.726014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:19.828202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679a532-5098-4b29-a992-453acbecbb33 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:19.868200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:19.907330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:19.907958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:19.908249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:19.908529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:19.910236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:20.023468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:20.024904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f552e-0bc4-48ee-a8fd-436058f4119b service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:20.045142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 733/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:22.175383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:22.175912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:22.175912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:22.177301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:22.182687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaffae5-22ad-46fe-b240-943aeae9d8cc None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:22.183124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaffae5-22ad-46fe-b240-943aeae9d8cc None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:22.183232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaffae5-22ad-46fe-b240-943aeae9d8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:22.249277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3aaffae5-22ad-46fe-b240-943aeae9d8cc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:22.250520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3aaffae5-22ad-46fe-b240-943aeae9d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:22.268752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:22.347603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:22.385408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:22.387025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:22.516297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f46c2-9f88-414b-a72f-199795eb6c77 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:22.581490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:22.588915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:22.590148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:22.590430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:22.590704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:22.592325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:22.700729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:22.702256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aae090-47e9-4266-8d69-4deaa5f1704d service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:22.723435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 733/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:24.729000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:24.729568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:24.729669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:24.730997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:24.736229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077fac4-6491-4a3b-9967-d68294f32794 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:24.736640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077fac4-6491-4a3b-9967-d68294f32794 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:24.736798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077fac4-6491-4a3b-9967-d68294f32794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:24.802317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6077fac4-6491-4a3b-9967-d68294f32794 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:24.803505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6077fac4-6491-4a3b-9967-d68294f32794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:24.856954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:24.906585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:24.944257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:24.945792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:25.051928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338251a5-0904-43d7-8fc2-8f7bb2b3e749 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:25.098659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:25.125844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:25.126505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:25.126810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:25.127089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:25.128744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:25.236799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:25.238279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef99597-4224-49ec-91f5-43d9c5da521e service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:25.256685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 736/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:34.904736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=22 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:35.254466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=66 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:44.919801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf8e4-3594-4651-8b69-82e0585a39df None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:44.920480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf8e4-3594-4651-8b69-82e0585a39df None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:44.920480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf8e4-3594-4651-8b69-82e0585a39df None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:44.921749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 735/1471] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:44.926838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f41a-202a-46dd-8818-6c6a7a690a71 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:44.927277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f41a-202a-46dd-8818-6c6a7a690a71 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:44.927478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f41a-202a-46dd-8818-6c6a7a690a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:44.994436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-80b6f41a-202a-46dd-8818-6c6a7a690a71 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:44.996239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-80b6f41a-202a-46dd-8818-6c6a7a690a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:45.095758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 737/1472] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:45.184073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:45.184387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:45.184661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:45.186075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 736/1473] 10.222.0.66 () {54 vars in 966 bytes} [Fri Jul 29 12:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:45.190729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21483-612a-49d3-83ca-af54a0456b59 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:45.191015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21483-612a-49d3-83ca-af54a0456b59 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:45.191289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc21483-612a-49d3-83ca-af54a0456b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:45.267134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-bdc21483-612a-49d3-83ca-af54a0456b59 None None] MFA Rules not processed for user `11a9dc108d68404ba926c566516e7812`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:45.268331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bdc21483-612a-49d3-83ca-af54a0456b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:45.288281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:45.344084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 738/1474] 10.222.0.66 () {58 vars in 1063 bytes} [Fri Jul 29 12:44:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:45.351593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:45.353089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:45.455296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1ee08-af2b-4b1b-a271-2122888823c3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:45.532454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:45.533273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:45.533551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:45.533829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:45.535471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:45.589715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:45.643949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:45.645387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c23b3-8486-41f8-8724-7f57a1c6a9c1 service glance] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:45.663367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 737/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:47.740711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:47.741284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:47.741284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:47.742708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 739/1476] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:47.748495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9712b8-7ba5-4bcc-b32d-95a375b7f8cc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:47.748880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9712b8-7ba5-4bcc-b32d-95a375b7f8cc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:47.749079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9712b8-7ba5-4bcc-b32d-95a375b7f8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:47.815841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-2b9712b8-7ba5-4bcc-b32d-95a375b7f8cc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:47.817053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2b9712b8-7ba5-4bcc-b32d-95a375b7f8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:47.914821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 738/1477] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:47.938689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:47.940218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:48.065577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8188cc-fc57-4e70-878e-3601fea1fd7b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:48.114260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:48.160325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:48.160937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:48.161219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:48.161494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:48.162219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:48.164007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:44:48.164007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:44:48.164007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:44:48.165154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3491d8-edab-4a90-8260-9cdcc032e373 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:48.184549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 740/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 29 12:44:47 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:44:50.266032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:50.266533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:50.266628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:50.267996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 739/1479] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:50.273381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec863b8-76a5-48d6-bbfa-80b5167a43b9 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:50.273909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec863b8-76a5-48d6-bbfa-80b5167a43b9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:50.274012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec863b8-76a5-48d6-bbfa-80b5167a43b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:50.344440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fec863b8-76a5-48d6-bbfa-80b5167a43b9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:50.345670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fec863b8-76a5-48d6-bbfa-80b5167a43b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:50.441422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 741/1480] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:50.485597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:50.487302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:50.497731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:50.637739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52943b05-1bbb-4466-856c-90171a2630ac None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:50.718968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:50.753257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:50.753909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:50.754190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:50.754461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:50.756099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:50.844574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:50.846185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826955a9-1742-434e-b042-77548ce4e59b service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:50.864449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 740/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:53.071440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:53.071983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:53.072106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:53.074024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 742/1482] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:53.079341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76d26-6643-413d-9fe0-1754499427bd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:53.079766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76d26-6643-413d-9fe0-1754499427bd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:53.079900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76d26-6643-413d-9fe0-1754499427bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:53.149534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-25b76d26-6643-413d-9fe0-1754499427bd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:53.150827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-25b76d26-6643-413d-9fe0-1754499427bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:53.197773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:53.257939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 741/1483] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:53.294639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:53.296197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:53.320333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:53.402082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1da37-d627-461f-9cd8-9448f310329a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:53.474817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:53.475436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:53.475711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:53.475981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:53.477583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:53.557493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:53.593343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:53.594828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17866dc0-c320-48f9-ae42-3042106a5f2f service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:53.614905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 743/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:55.672410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:55.672973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:55.672973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:55.674358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 742/1485] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:55.679492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8561-8a86-4d54-ac35-7fafa73417cf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:55.679942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8561-8a86-4d54-ac35-7fafa73417cf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:55.680083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8561-8a86-4d54-ac35-7fafa73417cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:55.869295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a84c8561-8a86-4d54-ac35-7fafa73417cf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:55.870560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a84c8561-8a86-4d54-ac35-7fafa73417cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:55.966033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 744/1486] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:56.004601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:56.006328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:56.111631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3783c5-7eaf-4791-840b-24dc5ca64205 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:56.184285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:56.186210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:56.186918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:56.187406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:56.187884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:56.189781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:56.305342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:56.307360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb183551-594a-4f1f-80c2-d5cfd1a12d64 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:56.325496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 743/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:44:58.435185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:58.435734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:58.435734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:58.437048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 745/1488] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:44:58.442346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf9960-d097-45f6-96d6-bba605b28310 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:58.442760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf9960-d097-45f6-96d6-bba605b28310 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:58.442897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf9960-d097-45f6-96d6-bba605b28310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:58.507562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-88cf9960-d097-45f6-96d6-bba605b28310 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:44:58.508747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-88cf9960-d097-45f6-96d6-bba605b28310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:58.560683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:58.604742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 744/1489] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:44:58.644692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:44:58.646257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:58.683229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:58.751876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab685c50-1343-4338-84d8-9105b5684aac None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:44:58.824829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:44:58.825440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:44:58.825730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:44:58.826039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:44:58.827838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:44:58.911464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:44:58.943638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:44:58.945064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cbfde-0427-4326-bc70-0606c5151fed service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:44:58.963642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 746/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:45:01.169285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:45:01.169923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:45:01.169923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:45:01.171204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 745/1491] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 29 12:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:45:01.176440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119aaa-18a9-4a29-8a6e-8c322f7325f8 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:45:01.176852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119aaa-18a9-4a29-8a6e-8c322f7325f8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:45:01.176988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119aaa-18a9-4a29-8a6e-8c322f7325f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:45:01.242418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-5a119aaa-18a9-4a29-8a6e-8c322f7325f8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:45:01.243620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5a119aaa-18a9-4a29-8a6e-8c322f7325f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:45:01.339086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 747/1492] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 29 12:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:45:01.379944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:45:01.381732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:45:01.486654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad746326-38b6-490d-ab56-215fe751c72c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:45:01.559509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:45:01.560116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:45:01.560394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:45:01.560667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:45:01.562305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:45:01.571233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:45:01.669080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:45:01.670470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b33660-fa22-478f-b6fe-894420c2d4df service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:45:01.688530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 746/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:45:11.337272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=82 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:45:11.686282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=50 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:29.133506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:29.133998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:29.134105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:29.206816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:46:29.208003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:29.302160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 748/1494] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:46:29.310102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:29.310598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:29.310701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:29.385861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:46:29.386999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:29.482801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 747/1495] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:46:29.487378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:29.488579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:29.500775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a6441-6948-4a9a-a837-38324f4a9de5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:29.564636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:29.599320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:29.599935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:29.600214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:29.600488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:29.601247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:46:29.601552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:29.602595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:29.602595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:46:29.602595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:46:29.603474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:29.614426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 749/1496] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:46:29.623124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:29.632805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7d070-8fe9-4645-a1c9-8a8bef4d8ebd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:29.692184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:29.728014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:29.728630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:29.728903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:29.729175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:29.729849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:29.731602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:29.731602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:46:29.731602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:46:29.732781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:29.787812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 748/1497] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:46:29 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:46:29.793093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:29.795267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:29.830001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42adb-2150-48f5-8fde-ee7291088ed5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:29.924575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:29.925186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:29.925461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:29.925728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:29.926419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:29.927428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:29.927428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:46:29.927428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:46:29.928607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:29.970005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 750/1498] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:46:29 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:46:29.974263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:29.976337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.037596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b6e30-ce34-44a7-9ad4-c3671599a98a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:30.122431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:30.134228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:30.134849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.135131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.135404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.136371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:30.137504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:30.137504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:46:30.137504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:46:30.138772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:30.151637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 749/1499] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:46:30.155816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:30.157336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.169042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33238d-e2ef-4986-b83d-7b587823eade admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:30.194319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:30.265631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:30.266275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.266551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.266816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.267567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:30.268916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:30.268916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:46:30.268916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:46:30.269793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77486-6930-4735-b342-5e9a0fb4042a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:30.281924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 751/1500] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:46:30.286242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:30.287780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.328064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f075c73-b126-4862-b0db-fd487406c298 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:30.424883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:30.425548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.425853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.426132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] PATH_INFO: `/v3/projects/6664f144526e4b05a942ffca15c36aa8/users/6d80b97bd8824c97b8b1b2ca709db286/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.448750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6664f144526e4b05a942ffca15c36aa8, user_id=6d80b97bd8824c97b8b1b2ca709db286, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:30.449770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:30.449770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:46:30.449770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:46:30.451767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f61b0-039e-44f5-ae4a-a19b3e1b72c7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:30.476234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 750/1501] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:46:30 2022] PUT /identity/v3/projects/6664f144526e4b05a942ffca15c36aa8/users/6d80b97bd8824c97b8b1b2ca709db286/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:46:30.488498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adb93d-3f62-4f6e-9814-c1bac0d81ae1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.488793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adb93d-3f62-4f6e-9814-c1bac0d81ae1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.489078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adb93d-3f62-4f6e-9814-c1bac0d81ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.547346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-75adb93d-3f62-4f6e-9814-c1bac0d81ae1 None None] MFA Rules not processed for user `6d80b97bd8824c97b8b1b2ca709db286`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:46:30.548609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-75adb93d-3f62-4f6e-9814-c1bac0d81ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.630545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 752/1502] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:46:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:46:30.644132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.644447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.644725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.646162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 751/1503] 10.222.0.66 () {54 vars in 964 bytes} [Fri Jul 29 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:46:30.650819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb39fc-f09e-44c5-9682-2947022de092 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.651115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb39fc-f09e-44c5-9682-2947022de092 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:30.651393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb39fc-f09e-44c5-9682-2947022de092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:30.683783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:30.731318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2acb39fc-f09e-44c5-9682-2947022de092 None None] MFA Rules not processed for user `b1a7571787a4415fb254a652062e50f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:46:30.732534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2acb39fc-f09e-44c5-9682-2947022de092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.808157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 753/1504] 10.222.0.66 () {58 vars in 1061 bytes} [Fri Jul 29 12:46:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:46:30.816112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:30.817669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:30.868649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:30.924419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f071-8874-425b-be52-3e6fd697778c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:30.998849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:30.999467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:30.999757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:31.000022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:31.001646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:31.101739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:31.103138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:31.107099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:31.122448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 752/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:46:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:46:31.170198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fc84-b9f5-4492-8353-8453be897242 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:31.170620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fc84-b9f5-4492-8353-8453be897242 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:31.170768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fc84-b9f5-4492-8353-8453be897242 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:31.172474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 754/1506] 10.222.0.66 () {52 vars in 830 bytes} [Fri Jul 29 12:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:46:31.177019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:31.178259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:31.277273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42cc4-7aea-437e-8963-f20b19190cfb service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:31.372987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:31.373582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:31.373851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:31.374131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] PATH_INFO: `/v3/users/6d80b97bd8824c97b8b1b2ca709db286` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:31.386223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d80b97bd8824c97b8b1b2ca709db286)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:31.387253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:31.387253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:46:31.387253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:46:31.388468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:31.398218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:31.453078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d80b97bd8824c97b8b1b2ca709db286'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:31.465495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d80b97bd8824c97b8b1b2ca709db286'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:31.465992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d80b97bd8824c97b8b1b2ca709db286'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:31.485297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d80b97bd8824c97b8b1b2ca709db286'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:31.485863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d80b97bd8824c97b8b1b2ca709db286'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:31.500663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 753/1507] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:46:31 2022] DELETE /identity/v3/users/6d80b97bd8824c97b8b1b2ca709db286 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:46:31.505059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:31.505446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:31.505733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:31.563142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:46:31.564460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:31.638361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:31.651429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 755/1508] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:46:31.660921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:31.662173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:31.765327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075070-db86-4d92-bc0b-f20c57d63196 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:31.824006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:31.845693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:31.846350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:31.846637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:31.846915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:31.848636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:31.954959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:31.956351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dce554-2c0e-4943-b864-09a8a2c6233e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:31.975379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 754/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:46:32.335662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:46:32.336880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:46:32.348682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8edc76-7adc-44b6-b612-5e6e23fb9948 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:46:32.455885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:46:32.456531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:46:32.456822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:46:32.457097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] PATH_INFO: `/v3/projects/6664f144526e4b05a942ffca15c36aa8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:46:32.468424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6664f144526e4b05a942ffca15c36aa8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:46:32.469657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:46:32.469657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:46:32.469657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:46:32.470886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:46:32.530835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6664f144526e4b05a942ffca15c36aa8', 'request_id': 'req-8ca1f175-200b-4deb-bfe1-35d13793676e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:32.536875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6664f144526e4b05a942ffca15c36aa8', 'request_id': 'req-8ca1f175-200b-4deb-bfe1-35d13793676e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:32.546726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6664f144526e4b05a942ffca15c36aa8', 'request_id': 'req-8ca1f175-200b-4deb-bfe1-35d13793676e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:32.577330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] The token cache is being invalidate because project 6664f144526e4b05a942ffca15c36aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:46:32.577621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:32.578193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8ca1f175-200b-4deb-bfe1-35d13793676e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:46:32.579553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 756/1510] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:46:32 2022] DELETE /identity/v3/projects/6664f144526e4b05a942ffca15c36aa8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:46:41.973162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=62 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:46:42.575331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:48:12.135652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:12.136185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:12.136185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:12.137626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:12.142566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1107b-090e-4701-a37f-25919727479d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:12.143026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1107b-090e-4701-a37f-25919727479d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:12.143161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1107b-090e-4701-a37f-25919727479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:12.226537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-0fd1107b-090e-4701-a37f-25919727479d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:12.227845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0fd1107b-090e-4701-a37f-25919727479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:12.335298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:48:12.490696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:12.492829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:12.611338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd8b0f-85f0-4bbc-869f-141d3766f23d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:12.618489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:12.619207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:12.619524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:12.619833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:12.628652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:12.630318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83e9da-e41b-428c-a7bc-321f00379f06 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:12.640481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 756/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:13.467300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bd06c-9d7e-4468-96bb-3e3ab0584ee5 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:13.467871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bd06c-9d7e-4468-96bb-3e3ab0584ee5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:13.467871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bd06c-9d7e-4468-96bb-3e3ab0584ee5 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:13.469160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:13.474303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee4429-f0b1-4587-8304-97cc7766715f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:13.474702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee4429-f0b1-4587-8304-97cc7766715f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:13.474864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee4429-f0b1-4587-8304-97cc7766715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:13.494559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c4ee4429-f0b1-4587-8304-97cc7766715f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:13.495746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c4ee4429-f0b1-4587-8304-97cc7766715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:13.503743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:13.654036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bd06c-9d7e-4468-96bb-3e3ab0584ee5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:13.668715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bd06c-9d7e-4468-96bb-3e3ab0584ee5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:13.674171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:13.674773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:13.675053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:13.675331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:13.683326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:13.684905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a35a03-0497-40b1-a6cb-4f5c06226423 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:13.694301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 759/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:14.843700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e4fc-62cb-4758-93db-33a6d7b6ef60 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:14.844243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e4fc-62cb-4758-93db-33a6d7b6ef60 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:14.844615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49e4fc-62cb-4758-93db-33a6d7b6ef60 None None] PATH_INFO: `/v3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:14.845951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 758/1517] 10.222.0.45 () {54 vars in 925 bytes} [Fri Jul 29 12:48:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:48:14.852896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec46e-c435-42b6-ae53-6f892001a806 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:14.853339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec46e-c435-42b6-ae53-6f892001a806 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:14.853443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec46e-c435-42b6-ae53-6f892001a806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:14.912135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-22aec46e-c435-42b6-ae53-6f892001a806 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:14.913331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-22aec46e-c435-42b6-ae53-6f892001a806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:14.929926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:48:14.984831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 760/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Jul 29 12:48:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:18.806582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d88f-9dd0-49be-9ad6-0e4963664bc7 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:18.807129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d88f-9dd0-49be-9ad6-0e4963664bc7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:18.807350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d88f-9dd0-49be-9ad6-0e4963664bc7 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:18.809369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 759/1519] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 29 12:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:18.815523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f79a6f-b32e-481d-9e11-d45b032640f6 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:18.815967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f79a6f-b32e-481d-9e11-d45b032640f6 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:18.816680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f79a6f-b32e-481d-9e11-d45b032640f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:18.836148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c3f79a6f-b32e-481d-9e11-d45b032640f6 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:18.837404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c3f79a6f-b32e-481d-9e11-d45b032640f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:18.846349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 761/1520] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 29 12:48:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:48:18.854156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d88f-9dd0-49be-9ad6-0e4963664bc7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:18.868012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d88f-9dd0-49be-9ad6-0e4963664bc7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:18.872943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:18.873544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:18.873848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:18.874120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:18.883047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:18.884619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e05d22-6027-433e-a54d-53823d5826e2 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:18.894611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 760/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:24.940065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0045a6-f706-4531-8cfb-55b1a02f9b67 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:24.940652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0045a6-f706-4531-8cfb-55b1a02f9b67 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:24.940652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0045a6-f706-4531-8cfb-55b1a02f9b67 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:24.942666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:24.947434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61964-35fe-42f1-bfde-1dd3a382a80b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:24.947831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61964-35fe-42f1-bfde-1dd3a382a80b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:24.947979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61964-35fe-42f1-bfde-1dd3a382a80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:24.967131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9ff61964-35fe-42f1-bfde-1dd3a382a80b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:24.968315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9ff61964-35fe-42f1-bfde-1dd3a382a80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:24.976375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:25.130653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0045a6-f706-4531-8cfb-55b1a02f9b67 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:25.147093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0045a6-f706-4531-8cfb-55b1a02f9b67 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:25.152361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:25.152961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:25.153237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:25.153558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:25.162470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:25.163874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfed30-a17f-4b85-9df9-e02f24fb6f3e service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:25.173483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 763/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:26.008805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32119-0ba2-4afe-91a7-02fb3d6eaec2 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:26.009381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32119-0ba2-4afe-91a7-02fb3d6eaec2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:26.009381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32119-0ba2-4afe-91a7-02fb3d6eaec2 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:26.010729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 12:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:26.015603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88900efd-6184-4478-983d-8bce8da1428d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:26.016023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88900efd-6184-4478-983d-8bce8da1428d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:26.016161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88900efd-6184-4478-983d-8bce8da1428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:26.035870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-88900efd-6184-4478-983d-8bce8da1428d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:26.037094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-88900efd-6184-4478-983d-8bce8da1428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:26.044923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 12:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:48:26.198264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa32119-0ba2-4afe-91a7-02fb3d6eaec2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:26.213495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa32119-0ba2-4afe-91a7-02fb3d6eaec2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:26.219002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:26.219872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:26.220154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:26.220427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:26.228631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:26.230250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a80ee-2cf8-47b2-a950-5f50cb1104ce service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:26.240405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 763/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:30.154301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:30.154798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:30.154906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:30.156221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:30.161262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f86-452a-4a0b-b381-241de6458621 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:30.161759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f86-452a-4a0b-b381-241de6458621 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:30.161889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f86-452a-4a0b-b381-241de6458621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:30.182570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-21321f86-452a-4a0b-b381-241de6458621 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:30.183750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-21321f86-452a-4a0b-b381-241de6458621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:30.192200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:30.265693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:30.267708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:30.348271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af2dac-a00b-4e65-95e6-3562e069f7f1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:30.351845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:30.352455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:30.352735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:30.353010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:30.361859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:30.363965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d770bfb-26ee-4566-b650-55a1142f9f1d service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:30.373754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 766/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:32.555375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc42f37-02ac-45c0-b577-5f5cb7e0d2fb None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:32.555958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc42f37-02ac-45c0-b577-5f5cb7e0d2fb None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:32.555958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc42f37-02ac-45c0-b577-5f5cb7e0d2fb None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:32.557390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:32.562601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506102-6dcf-4c1f-a80e-19cdb4b0b58d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:32.563020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506102-6dcf-4c1f-a80e-19cdb4b0b58d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:32.563209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506102-6dcf-4c1f-a80e-19cdb4b0b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:32.583543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-33506102-6dcf-4c1f-a80e-19cdb4b0b58d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:32.584654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-33506102-6dcf-4c1f-a80e-19cdb4b0b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:32.592364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:32.630394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc42f37-02ac-45c0-b577-5f5cb7e0d2fb None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:32.641068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc42f37-02ac-45c0-b577-5f5cb7e0d2fb None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:32.644678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:32.645335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:32.645617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:32.645925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:32.654844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:32.656277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960f31-d0be-407b-9094-1ec02ebe5d11 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:32.666293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 766/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:36.105716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:36.106287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:36.106287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:36.107956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:36.113629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7ddd-ee30-40ea-987d-ceb73aa0cddc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:36.114114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7ddd-ee30-40ea-987d-ceb73aa0cddc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:36.114228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7ddd-ee30-40ea-987d-ceb73aa0cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:36.135337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a60e7ddd-ee30-40ea-987d-ceb73aa0cddc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:36.136523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a60e7ddd-ee30-40ea-987d-ceb73aa0cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:36.144337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:36.448960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:36.450654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:36.515324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:48:36.528522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386763f-c3bc-4b44-b81f-0136fc3184d3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:36.532069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:36.532681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:36.532965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:36.533233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:36.542900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:36.544395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a484-3f4b-4b92-a8d9-131a3bca24f2 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:36.554847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 769/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:38.615943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:38.616523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:38.616523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:38.617921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 12:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:48:38.623519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2d8-5887-4f20-b2c4-fd1cb667d07c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:38.623983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2d8-5887-4f20-b2c4-fd1cb667d07c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:38.624118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2d8-5887-4f20-b2c4-fd1cb667d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:38.647586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f3dad2d8-5887-4f20-b2c4-fd1cb667d07c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:48:38.648774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3dad2d8-5887-4f20-b2c4-fd1cb667d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:38.656577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 12:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:48:38.961421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:48:38.963095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:48:38.976928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59ba22-3772-42e1-8c8e-0cd6e02bbead None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:48:38.980801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:48:38.981876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:48:38.982372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:48:38.982892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:48:38.996737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:48:38.997036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0efb277-401f-437a-827f-c56453ebfd9c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:48:39.007493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 769/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:48:48.642791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=11 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:48:49.002577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:07.745792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ee7e-d4e8-4d47-9e7f-6b90afcf4db0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.747015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ee7e-d4e8-4d47-9e7f-6b90afcf4db0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.747531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ee7e-d4e8-4d47-9e7f-6b90afcf4db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.779027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9702ee7e-d4e8-4d47-9e7f-6b90afcf4db0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:07.780646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9702ee7e-d4e8-4d47-9e7f-6b90afcf4db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:07.789226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 771/1540] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:07.799814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.800419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.800419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.833561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:07.834838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:07.839170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd7c-07bf-4c91-a485-415156653f77 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.839729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd7c-07bf-4c91-a485-415156653f77 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.844517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd7c-07bf-4c91-a485-415156653f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.847422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 770/1541] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:07.852492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:07.865339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453d915-15f3-4a6f-bd80-863873bf7f4e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:07.874524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:07.876029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.876489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.876931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.877847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:07.878221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d524cd7c-07bf-4c91-a485-415156653f77 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:07.878578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:07.879800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d524cd7c-07bf-4c91-a485-415156653f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:07.880170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:07.880170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:50:07.880170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:50:07.881429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:07.902638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 772/1542] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:07.908648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 771/1543] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:50:07.914431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caaa1d-da73-40c5-a73e-9c60e2588ba7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.914847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caaa1d-da73-40c5-a73e-9c60e2588ba7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.915230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caaa1d-da73-40c5-a73e-9c60e2588ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.918043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:07.935084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ca162-cc51-4100-9c19-c034a5d5cf9d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:07.941555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e9caaa1d-da73-40c5-a73e-9c60e2588ba7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:07.942881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e9caaa1d-da73-40c5-a73e-9c60e2588ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:07.945124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:07.945977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.946428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.946872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:07.947705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:07.949778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f416b-1380-4d80-82ee-a5083380c87d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:07.963504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 773/1544] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:07.964928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:07.965221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:07.965501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.012326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.013448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.038355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 774/1545] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.040252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 772/1546] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:07 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:08.041532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f6d8-6299-45bb-80e7-ebba755d3554 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.041850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f6d8-6299-45bb-80e7-ebba755d3554 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.042123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f6d8-6299-45bb-80e7-ebba755d3554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.045305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.060007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4786496-2142-45bd-9254-4d6856d14a18 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.079404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1cd0f6d8-6299-45bb-80e7-ebba755d3554 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.080614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1cd0f6d8-6299-45bb-80e7-ebba755d3554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.184345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.185014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.185331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.185610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.187219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.188364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.190013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.218264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 773/1547] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.221006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 775/1548] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.221006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.227277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed9674-71e6-4b68-9b70-4d14eb718bbf None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.227821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed9674-71e6-4b68-9b70-4d14eb718bbf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.228347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed9674-71e6-4b68-9b70-4d14eb718bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.240000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11abd-d980-4238-abf9-ecd8f923dbf3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.245620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.246483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.247028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.247487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.248363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.250647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fd6d1-c9c1-4c71-b8ef-b86adde6db7d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.254225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-eeed9674-71e6-4b68-9b70-4d14eb718bbf None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.255538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-eeed9674-71e6-4b68-9b70-4d14eb718bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.268536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 774/1549] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.270366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0274-1fd7-400f-8ec0-63ae4f0328d1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.270870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0274-1fd7-400f-8ec0-63ae4f0328d1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.271376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0274-1fd7-400f-8ec0-63ae4f0328d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.278105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 776/1550] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.279727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.280212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.280687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.301402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-925d0274-1fd7-400f-8ec0-63ae4f0328d1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.301638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.302685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-925d0274-1fd7-400f-8ec0-63ae4f0328d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.302855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.310504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 777/1551] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.313267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 775/1552] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.313267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f031-be08-46cf-af40-84bdf48c8a09 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.313267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f031-be08-46cf-af40-84bdf48c8a09 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.313267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f031-be08-46cf-af40-84bdf48c8a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.314618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.324326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b72ff7-b73b-4a26-acd6-262939cf75c1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.329450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.330090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.330375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.330651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.331310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.332320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:08.332320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:50:08.332320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:50:08.332672 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9946f031-be08-46cf-af40-84bdf48c8a09 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.333501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.333881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9946f031-be08-46cf-af40-84bdf48c8a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.341967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 776/1553] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.347763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3827e4b-20cf-41de-83ce-11493631bfa7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.347763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3827e4b-20cf-41de-83ce-11493631bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.347763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3827e4b-20cf-41de-83ce-11493631bfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.368022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c3827e4b-20cf-41de-83ce-11493631bfa7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.368258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 778/1554] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.368909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.369585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c3827e4b-20cf-41de-83ce-11493631bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.430662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 777/1555] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.431918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.432212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.432491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.433363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a59c-32e0-43f3-a840-3cbd6ecf8411 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.438589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.439215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.439975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.440362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.441092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.441413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.443073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.451607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.452820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.461967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 779/1556] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.461967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.466955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 778/1557] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.467766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.474351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3822-d159-4bdf-96e4-44b680541b38 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.477475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3f923-f9b4-4f22-b5ac-fa3e4375d7fd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.479236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.479873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.480185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.480457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.481188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.482167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.482770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.482887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624ab82-6b75-469d-ae42-fee683d39098 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.483053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.483333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.484027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.484857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.486489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214053d2-1fb8-469b-85aa-4d26902d946a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.494935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 780/1558] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:08.496125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.496408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.496678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.499128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 779/1559] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.500611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4ea6-8c8e-4c02-a551-f45c0ef05249 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.501191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4ea6-8c8e-4c02-a551-f45c0ef05249 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.501473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4ea6-8c8e-4c02-a551-f45c0ef05249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.506455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:08.518848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.519989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.521799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-450a4ea6-8c8e-4c02-a551-f45c0ef05249 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.522977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-450a4ea6-8c8e-4c02-a551-f45c0ef05249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.528153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 781/1560] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.528995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.531033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 780/1561] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.535050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.535336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.535609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.539297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9149-ca95-4855-b29f-8a315e8c6491 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.543813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.543979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:08.544418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.544719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.544989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.545747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.548406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e451-3c3b-4f93-a1f0-0ed79a00b83b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.556086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.557371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.566256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 781/1562] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.567076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.573532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 782/1563] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.574355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.574651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.574931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.577001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa91d19-5874-47b1-86f6-6d1027892ad5 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.581760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.582381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.582657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.582923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.583574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.585455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.594361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:08.595813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:08.603748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 783/1564] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:08.604529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.614051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24906ae2-819d-4413-978c-6e7a21c62b01 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.614230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 782/1565] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.614978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.673005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.673956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.674237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.674509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.675202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.676804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.677242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8e03-20d8-45bf-8b2b-b02027ae68e9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.681921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.682536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.682819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.683093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.683749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.685692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.686604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 784/1566] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:08.687399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.697902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c57b9-4ca1-4dde-b9b6-123486635084 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.702548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.703174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.703542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.703753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.704431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.704734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.706339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.712626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 783/1567] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.713450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.722588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 785/1568] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.723737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.735928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a58bd-d861-4a80-a4cc-bcc82c545a29 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.775609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecb8b-6e79-4a7c-93be-4101c4f8c048 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.780268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.780883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.781166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.782049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.782744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.783054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.784659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.792948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.793568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.793880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.794163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.794852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.795159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.795858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 784/1569] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.803518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.803645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.806716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687db0b0-6912-4d0c-956d-e52e99369f04 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.808975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 786/1570] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.809811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.815655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.816267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.816551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.816831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.817525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.819154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.821934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a4e-bbf8-4166-945a-ce4dd962cc3d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.826660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.827257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.827609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.827888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.828686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:08.828994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.829358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 785/1571] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:08.830158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.830654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.841430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c468dc-84e8-4689-9efd-fc03ccd1ee22 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.843595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 787/1572] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:08.844753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.851407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.851884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.852198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.852816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.853619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.855092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:08.855092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:50:08.855092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:50:08.856310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.858267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49310945-72c2-4941-8138-f962255005f3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.862916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.863568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.863850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.864129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] PATH_INFO: `/v3/projects/f9210aacd9bd4593a85429fe1768e28d/users/b3a4fdcc0dde49009f4e55d25af4da7c/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.879373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 786/1573] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.879625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9210aacd9bd4593a85429fe1768e28d, user_id=b3a4fdcc0dde49009f4e55d25af4da7c, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.880159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.880715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:08.880715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:50:08.880715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:50:08.883118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.892163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd372-4a18-487e-8bbf-71bea2021b28 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.898245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 788/1574] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:08 2022] PUT /identity/v3/projects/f9210aacd9bd4593a85429fe1768e28d/users/b3a4fdcc0dde49009f4e55d25af4da7c/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:08.899080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.912860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:08.917300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:08.952119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.952754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.953037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.953686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.954417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.956373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.965349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f925-8d53-4af0-abe8-15f1b02f9ac3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.970739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.971357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.971670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.971972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.972695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.974642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:08.978456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 787/1575] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:08.979248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:08.989312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0359-d19c-41f0-93e4-0fd4515f57e1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:08.994237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:08.994902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:08.996254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:08.996254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:08.996254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:08.998143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.005287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 789/1576] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.006242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.030919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 788/1577] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:08 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.030919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.076309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2ac4-f3ba-41bc-9f05-d4ce311b22a9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.089989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e67139-38fa-4cb8-931a-3c6e27e0fc13 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.094654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.095565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.095852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.096124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.096956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.098741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.108910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 789/1578] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.110258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.120557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda5235-b10e-4cb5-b0ba-0c301ccf4c70 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.125326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.125971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.126257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.126533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.127225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.129127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.137532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.138177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.138463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.138735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.139414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.141872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83113001-901c-402a-a09e-086be8e5c269 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.157030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 790/1579] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.157966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.170286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 790/1580] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.171659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.172445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.172750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.190965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:09.192174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:09.219457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:09.224084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add678a-4d4a-4854-b4c6-30f5c686c92c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.236858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:09.257555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 791/1581] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:09.261165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.268084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3c26-1109-4930-bc4a-52b6461f61e5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.273326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.273964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.274279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.274576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.275277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.276868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.285160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.285725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.286038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.286316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.287015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.287209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 792/1582] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.287988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.288608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.298338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3ec50-76fe-484a-b58a-f89264950c80 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.300335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 791/1583] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.300530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.303264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.303871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.304151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.304426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.305080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.306999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.311762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cea8a-36d8-477b-8404-2655e7ccaa1c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.316961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.317549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.317877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.318245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.318893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.320773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.330043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 793/1584] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.331289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.342019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb11675-89bd-4170-b8c4-569fd8026f49 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.344609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 792/1585] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.345700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.350339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.351053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.351336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.352130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] PATH_INFO: `/v3/projects/9cdcbcec39334733a5fa29ddc40c7a2d/users/1dd24e8c0d214a0bb62578f9834a40a2/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.355560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341b2c-ebaf-4cb9-a06d-67da7db4100f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.360733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.361340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.361598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.361899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.362544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.364446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.367273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdcbcec39334733a5fa29ddc40c7a2d, user_id=1dd24e8c0d214a0bb62578f9834a40a2, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.369922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.384506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 794/1586] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:09 2022] PUT /identity/v3/projects/9cdcbcec39334733a5fa29ddc40c7a2d/users/1dd24e8c0d214a0bb62578f9834a40a2/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:09.385489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.388887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 793/1587] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.390263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.447519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db58a3-867f-4842-91eb-dc9ccdc1a597 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.448258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0e6d-b099-47b4-88a7-defc36286e9b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.452775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.453407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.453688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.453991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.454647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.456581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.482397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 795/1588] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:09.482397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.490516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119322cc-7fe0-48e7-86c2-0fa2f7488798 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.495166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.495768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.496050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.496331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.497024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.498860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.503723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.504368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.504670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.504933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.509034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 796/1589] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.512692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.515717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.517167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.519050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ed2a-1fe4-4337-9e24-19e1594b41d2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.523641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.524221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.524504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.524780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.525458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.526780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 794/1590] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:09.527070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.527420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.536867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db696c6-7714-409a-a797-402f657b9666 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.539176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 797/1591] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.542835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.543011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.543453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.543738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.544003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.544697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.546317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.552785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c26e37-ec42-47db-9964-ec5512422393 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.557640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 795/1592] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.557792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.558139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.558323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.558432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.558776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.559419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.561318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a72ae-a02d-4216-a1e6-9faf43d51523 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.569841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3951-70b9-4def-ab5f-5852238ae516 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.573396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 798/1593] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.574689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.574979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.575257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.577507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.578800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.579090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.580112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.580818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.582475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.596790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 796/1594] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.596790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.597317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:09.597749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:09.610528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797ecd-ac68-439d-aec2-d3f2bbed06fa admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.610741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 799/1595] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:09.610741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.619282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f0bf2-ddcd-46fb-bf59-7700aedbf4d9 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.620969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.621617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.621959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.622248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.622940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.623963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.624541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.624663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.624899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.625815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] PATH_INFO: `/v3/projects/f4463aa83c704417b7662b80cda75ba8/users/6def97bed1a14e849f53a1a07906e111/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.639076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 797/1596] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.639076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.642157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:09.648911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4463aa83c704417b7662b80cda75ba8, user_id=6def97bed1a14e849f53a1a07906e111, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.649644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d13d82-3deb-4b8d-8873-f3b47ed2aff1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.652062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.656898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.657651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.657952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.658164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] PATH_INFO: `/v3/projects/d57fc58eac4149e780e4329e87582260/users/b4d57a4e83ab4310b5b3ba90f43c5ddc/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.668499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 800/1597] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:09 2022] PUT /identity/v3/projects/f4463aa83c704417b7662b80cda75ba8/users/6def97bed1a14e849f53a1a07906e111/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:09.669465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.676385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:09.677473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57fc58eac4149e780e4329e87582260, user_id=b4d57a4e83ab4310b5b3ba90f43c5ddc, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.681351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.702092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 798/1598] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:09 2022] PUT /identity/v3/projects/d57fc58eac4149e780e4329e87582260/users/b4d57a4e83ab4310b5b3ba90f43c5ddc/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:09.702852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.745555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ee5ef-3a9f-40b2-8f9a-d9b0f7f754b5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.783660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db306fb-e501-4bb8-8760-a135b53ccf77 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.789150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.789786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.790249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.790599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.791385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.793059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.804951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 799/1599] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.805903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.816163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d594f-8eab-4a1b-a490-0aa40bf07a2b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.822033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.822747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.823105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.823390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.824070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.824556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.824854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.825229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.825574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.825726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.826797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.829654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.837133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 800/1600] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.837800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.839518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:09.843523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 801/1601] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.844421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.854688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d9b6d-0001-4bdd-afca-cee40b450326 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.859999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.860712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.861065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.861414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.862221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.863956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d61e9-b532-47e8-a19b-5f5cd8e7a63e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.879134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 802/1602] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:09.879134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.879134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.879134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.907038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] MFA Rules not processed for user `6def97bed1a14e849f53a1a07906e111`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:09.908378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:09.916832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5234ca-7d3a-46d0-b6f8-f9988e922602 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:09.921232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:09.922513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.922804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.923080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.933799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:09.935292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff2106-3490-4234-b489-e0180bb396ac service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:09.946489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 801/1603] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:09.947552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:09.947838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:09.948172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:09.972561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:09.973776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:09.983496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 803/1604] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:09.983496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.996760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 802/1605] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:09.997918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:09.999989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a7279-df8f-4adb-8acf-0d1ef476349e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.007143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.008930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.009404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.009937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] PATH_INFO: `/v3/projects/367b19dc4a364576b869cef1931f0356/users/8c36da943dc24f848dab5f628d7083a0/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.010644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac992a2a-1da6-4a6c-8693-55030aed7549 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.018988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.019908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.020397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.020898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] PATH_INFO: `/v3/projects/dc5920aa5d9c4b8393148fc8ae6db786/users/5b65afab1e5e44cbbb53fc99a4590178/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.036781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=367b19dc4a364576b869cef1931f0356, user_id=8c36da943dc24f848dab5f628d7083a0, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.040745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.046157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5920aa5d9c4b8393148fc8ae6db786, user_id=5b65afab1e5e44cbbb53fc99a4590178, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.050891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.064927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 804/1606] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:09 2022] PUT /identity/v3/projects/367b19dc4a364576b869cef1931f0356/users/8c36da943dc24f848dab5f628d7083a0/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:10.074053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.074053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:10.074420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 803/1607] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:09 2022] PUT /identity/v3/projects/dc5920aa5d9c4b8393148fc8ae6db786/users/5b65afab1e5e44cbbb53fc99a4590178/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:10.074420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.148891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:10.172462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0ca02-bc31-4b0f-857e-03511a58a79a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.186244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71943791-cbac-42c9-a790-840bb912741e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.194496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.195549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.196150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.196611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] PATH_INFO: `/v3/projects/89441418bb3d4ff6ad1a0b466e69cfb6/users/e7ee2c3480f8483bb76ecc686215720b/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.222567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89441418bb3d4ff6ad1a0b466e69cfb6, user_id=e7ee2c3480f8483bb76ecc686215720b, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.226703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.238326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.239203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.239582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.239959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] PATH_INFO: `/v3/projects/3be5d64be552499d888f5d6bc6ad8414/users/52feac75e3484fe09e2b2af895640995/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.249640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 804/1608] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:10 2022] PUT /identity/v3/projects/89441418bb3d4ff6ad1a0b466e69cfb6/users/e7ee2c3480f8483bb76ecc686215720b/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:10.251546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.258164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be5d64be552499d888f5d6bc6ad8414, user_id=52feac75e3484fe09e2b2af895640995, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.261724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.292460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 805/1609] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:10 2022] PUT /identity/v3/projects/3be5d64be552499d888f5d6bc6ad8414/users/52feac75e3484fe09e2b2af895640995/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:10.293545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.379450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbcfb9-f282-438b-8dac-cd6755279b32 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.379450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.379450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.379450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.380005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.428472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54caf13d-423a-4eee-b3f6-bfe9b013e133 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.445130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:10.492669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.494677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa1c0-d77d-4d11-a4ab-71196e476821 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.511811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 806/1610] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:10.513480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58c3e-47b8-4b86-a02d-ee1cf10ea44a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.514133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58c3e-47b8-4b86-a02d-ee1cf10ea44a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.514422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58c3e-47b8-4b86-a02d-ee1cf10ea44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.548703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.549496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.550136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.550671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.551443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.553666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d11065-33b1-4854-8cab-7a1356425bd7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.556458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b7b58c3e-47b8-4b86-a02d-ee1cf10ea44a None None] MFA Rules not processed for user `8c36da943dc24f848dab5f628d7083a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:10.558334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b7b58c3e-47b8-4b86-a02d-ee1cf10ea44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:10.606154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 805/1611] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/projects => generated 373 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:10.608873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76bd2-48d3-4427-ad11-c3d8e65e0c1c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.610129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76bd2-48d3-4427-ad11-c3d8e65e0c1c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.610613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a76bd2-48d3-4427-ad11-c3d8e65e0c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.651856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-00a76bd2-48d3-4427-ad11-c3d8e65e0c1c None None] MFA Rules not processed for user `5b65afab1e5e44cbbb53fc99a4590178`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:10.654268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-00a76bd2-48d3-4427-ad11-c3d8e65e0c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:10.657275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 807/1612] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:10.658756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.659145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.659635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.684902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:10.687306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:10.715747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:10.772817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 806/1613] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:10.772817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.772817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.772817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.807194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 808/1614] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:10.809799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.809951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] MFA Rules not processed for user `52feac75e3484fe09e2b2af895640995`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:10.811214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:10.840507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e844-070b-4a33-81b4-4baadb254e42 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.865076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.865669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.865989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.866242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.866940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.868986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8ff5d-2d06-4ba3-befd-807dcc6ae816 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.905167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 809/1615] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:10.905167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228af1-e5a0-43af-97d2-885cf89fc204 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.905167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228af1-e5a0-43af-97d2-885cf89fc204 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.905480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff228af1-e5a0-43af-97d2-885cf89fc204 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.908749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 810/1616] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 29 12:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:10.917471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58a5a-3970-4fd1-9a05-52c0d04359f7 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.917846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58a5a-3970-4fd1-9a05-52c0d04359f7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.918217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58a5a-3970-4fd1-9a05-52c0d04359f7 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.919922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 811/1617] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 29 12:50:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:10.920723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58a5a-3970-4fd1-9a05-52c0d04359f7 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.928936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 807/1618] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:10.930012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:10.943838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc5c93-7538-4306-8728-b13bd68b51b5 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:10.949169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:10.949990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.950333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.950709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.951687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:10.953461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282b1e-eb98-446d-ae2c-89bcba045529 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:10.969388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 808/1619] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:10.969388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:10.969388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:10.969388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:10.995947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58a5a-3970-4fd1-9a05-52c0d04359f7 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.003130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.004112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.004982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.004982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.021492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:11.022952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:11.028778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.030640 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7586959-47d6-413d-95bf-5663ffaa0f54 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.037328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:11.047882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 812/1620] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:11.050501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.050978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.051417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.080327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:11.083294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:11.153338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 809/1621] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 29 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:11.154501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.170940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf34c6-de3b-4899-b385-6e3c3bcb7984 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.178545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.179236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.179568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.179998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] PATH_INFO: `/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0/users/ec7465e03c3643678d5181b79d128a4a/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.205251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e57f03d6e241a397bd4dbe3e96d5d0, user_id=ec7465e03c3643678d5181b79d128a4a, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.208810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.236682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 813/1622] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 29 12:50:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:11.238009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.252143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 810/1623] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:11 2022] PUT /identity/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0/users/ec7465e03c3643678d5181b79d128a4a/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:11.253747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.266721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4b812-69c1-4d6b-955e-653271bf59a9 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.292646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:11.367618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.368311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.368595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.368871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.439304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964f74-35f1-48f4-aafa-6d2fdcd6f15c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.452342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.453532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.453860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.454206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.507260 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.509288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.529934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 814/1624] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:11.531395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.599102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.600767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.613892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:11.620973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 811/1625] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:11.711496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ab0d-b65d-4f19-9da5-985c0cf99ec4 service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.724353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.724999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.725280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.725554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.729488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.736733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.763188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 815/1626] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:11.768590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.785601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243633e-59ff-4bce-92bb-3148aaa350ed service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.803391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:11.807225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.807518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.807803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] PATH_INFO: `/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0/users/ec7465e03c3643678d5181b79d128a4a/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.810901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e57f03d6e241a397bd4dbe3e96d5d0, user_id=ec7465e03c3643678d5181b79d128a4a, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:11.834636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d63c3-bc1b-4cc0-a2af-b8b32242673f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:11.854064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:11.862333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 812/1627] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:11 2022] PUT /identity/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0/users/ec7465e03c3643678d5181b79d128a4a/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:11.873317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:11.873703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:11.874104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:11.887007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99d538-a3a8-417f-8897-c1d11a6daa28 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:11.941376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] MFA Rules not processed for user `ec7465e03c3643678d5181b79d128a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:11.943149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:12.034174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:12.034806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.035083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.035353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.108536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 813/1628] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:12.128941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:12.131141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:12.131323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e1584-75e0-47e1-8dc8-d661801cc5d2 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:12.144702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 816/1629] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:12.173571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:12.185590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585df07-1c24-4fff-ab85-71bf7f376d46 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.185590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585df07-1c24-4fff-ab85-71bf7f376d46 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.185590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585df07-1c24-4fff-ab85-71bf7f376d46 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.187808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 817/1630] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 29 12:50:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:50:12.197602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b888ea-f6e4-4e09-886d-562dae5e95c3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.198008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b888ea-f6e4-4e09-886d-562dae5e95c3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.198296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b888ea-f6e4-4e09-886d-562dae5e95c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.240046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f4b888ea-f6e4-4e09-886d-562dae5e95c3 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:12.243877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f4b888ea-f6e4-4e09-886d-562dae5e95c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:12.254930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0080a-dec5-4006-ae69-c84301310624 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:12.260013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:12.260772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.261119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.261666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.267784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 818/1631] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 29 12:50:12.270392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e560c-9946-4475-96f9-12e8649653ce None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.276120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e560c-9946-4475-96f9-12e8649653ce None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.276479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e560c-9946-4475-96f9-12e8649653ce None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.278081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 819/1632] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 29 12:50:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:12.283935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:12.284254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.284603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.284995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.286658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:12.303605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 814/1633] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:12.305039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:12.320314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:12.321114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cca1f-f54a-42ad-9444-2ffc87e3a177 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:12.323132 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:12.326105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:12.327122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.327751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.328264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.347117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 820/1634] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:12.359046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:12.360921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:12.362965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:12.388194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218b21a-03ba-49dc-a1f2-3c0e89ddfb81 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:12.393732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:12.401620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 815/1635] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:12.405512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:12.406838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:12.408092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:12.430288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:12.432194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:12.449322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 821/1636] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:13.465512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:13.484144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6319ff-c13b-4b92-b207-a9e964b98143 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:13.501299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:13.502318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:13.502765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:13.503465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:13.658517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:13.660966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f9e00-8a18-4401-997d-f49af84d4f02 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:13.688331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 816/1637] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:14.063918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:14.149319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:14.206049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520a084-58ac-4352-a4a6-3d0b73549b10 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:14.216017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:14.218473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:14.218793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:14.219069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] PATH_INFO: `/v3/users/8c36da943dc24f848dab5f628d7083a0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:14.221110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c36da943dc24f848dab5f628d7083a0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:14.223902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: WARNING py.warnings [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:14.223902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: warnings.warn(msg) Jul 29 12:50:14.223902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: Jul 29 12:50:14.225164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:14.367435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c36da943dc24f848dab5f628d7083a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:14.370433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c36da943dc24f848dab5f628d7083a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:14.386700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c36da943dc24f848dab5f628d7083a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:14.443029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c36da943dc24f848dab5f628d7083a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:14.467157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c36da943dc24f848dab5f628d7083a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:14.478241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 822/1638] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:14 2022] DELETE /identity/v3/users/8c36da943dc24f848dab5f628d7083a0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:14.487611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:14.489849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:14.490553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:14.543252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:14.545199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:14.714754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 817/1639] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:14.806603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:14.808055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:14.961881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307dd9b-f4e0-45cc-b5d6-7953f8c1d6b2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:14.961881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:14.961881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:14.961881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:14.966715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:14.981996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:14.987342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925135-5217-4aad-bb8c-d4dfa2685238 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:15.015056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 823/1640] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:16.482720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:16.485010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:16.520580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d8124-1ee6-4a05-ba00-395f7dee9640 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:16.532684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:16.533887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:16.534171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:16.534445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] PATH_INFO: `/v3/projects/367b19dc4a364576b869cef1931f0356` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:16.538403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367b19dc4a364576b869cef1931f0356)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:16.540930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: WARNING py.warnings [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 12:50:16.540930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: warnings.warn(msg) Jul 29 12:50:16.540930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: Jul 29 12:50:16.542311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:16.676835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:16.685083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367b19dc4a364576b869cef1931f0356', 'request_id': 'req-f34b34fe-8075-4c87-9199-36274a3abe02'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:16.720780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367b19dc4a364576b869cef1931f0356', 'request_id': 'req-f34b34fe-8075-4c87-9199-36274a3abe02'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:16.735903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367b19dc4a364576b869cef1931f0356', 'request_id': 'req-f34b34fe-8075-4c87-9199-36274a3abe02'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:16.756821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] The token cache is being invalidate because project 367b19dc4a364576b869cef1931f0356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:16.757116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:16.771098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f34b34fe-8075-4c87-9199-36274a3abe02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:16.777126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 818/1641] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:16 2022] DELETE /identity/v3/projects/367b19dc4a364576b869cef1931f0356 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:16.781190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee030d-6198-442d-b200-195191373b40 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:16.782642 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee030d-6198-442d-b200-195191373b40 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:16.783083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee030d-6198-442d-b200-195191373b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:16.846264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-bfee030d-6198-442d-b200-195191373b40 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:16.848215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bfee030d-6198-442d-b200-195191373b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:16.978562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:17.003397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 824/1642] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:17.011508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.011799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.012077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.048805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:17.053858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:17.082964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 819/1643] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:17.088160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee030d-6198-442d-b200-195191373b40 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.122216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee030d-6198-442d-b200-195191373b40 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.141324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.142339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.145532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.146072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.147001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:17.147494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.149297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.194237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 825/1644] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:17.199432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.211868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40d72c-1650-4db9-8364-05fda68f72d9 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.237660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.239441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.239934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.240405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.241279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.244037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.323330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 820/1645] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:17 2022] POST /identity/v3/projects => generated 403 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:17.328157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.347283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94ae1-d2c8-4075-aeac-28098fcd5e9b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.485890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.486740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.487030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.487844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.488482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.490470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.541814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.547979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 826/1646] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:17 2022] POST /identity/v3/users => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:17.553093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6243-f6e6-4030-acc4-e6ede303445d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.572161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.572828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.573112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.573384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.574132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.575756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.603440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 821/1647] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:17.607703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.625392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bafcd-2a25-44e3-bbfd-28612cba75e7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.634929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.635586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.635868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.636144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.636862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.639363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b63c5-5465-4947-ae53-e1f6e9bb9594 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.667949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:17.679260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0d521-2616-4328-b6cc-738e1fcf83ee admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:17.680217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 827/1648] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:17.697858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:17.698596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.698881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.699159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] PATH_INFO: `/v3/projects/308481ff3a9b480eab9fed402f5224aa/users/e8b48e6bcdbf421ab7d53eaaa7f4e36b/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.746241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308481ff3a9b480eab9fed402f5224aa, user_id=e8b48e6bcdbf421ab7d53eaaa7f4e36b, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:17.751517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:17.777970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 822/1649] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:17 2022] PUT /identity/v3/projects/308481ff3a9b480eab9fed402f5224aa/users/e8b48e6bcdbf421ab7d53eaaa7f4e36b/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:17.789112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:17.789625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:17.789726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:17.838561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] MFA Rules not processed for user `e8b48e6bcdbf421ab7d53eaaa7f4e36b`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:17.839811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:17.951100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 828/1650] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:17.951286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:18.119357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb794a-d786-45e0-b2d9-8e4452fddb85 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:18.133225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:18.134512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:18.134937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:18.135222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:18.135993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:18.141029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:18.217233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 823/1651] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:17 2022] POST /identity/v3/projects => generated 405 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:18.219581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:18.257997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e679b-f666-44b8-849b-0f1ccdf040db None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:18.553971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:18.578610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:18.579255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:18.579536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:18.579806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:18.580669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:18.582522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:18.619837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 829/1652] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:18 2022] POST /identity/v3/users => generated 309 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:18.623716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:18.639440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956e61a-eac5-46b3-ac6f-617f7514b812 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:18.646921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:18.647946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:18.648430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:18.648924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:18.649858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:18.656807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:18.677421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 824/1653] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:18.681849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:18.701170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074216aa-9bff-49ac-9089-e600cbfd3aec admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:18.711591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:18.712927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:18.713409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:18.714810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] PATH_INFO: `/v3/projects/12ba6569a3614ee0b4213ba2a68261ca/users/919208e7469849fb8c387f892d064bd6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:18.751201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ba6569a3614ee0b4213ba2a68261ca, user_id=919208e7469849fb8c387f892d064bd6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:18.757201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:18.782195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:18.782785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 830/1654] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:18 2022] PUT /identity/v3/projects/12ba6569a3614ee0b4213ba2a68261ca/users/919208e7469849fb8c387f892d064bd6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:18.822718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061df2e-23e7-4f69-ab9b-f2fd895248db admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:18.964168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:18.995649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:18.996632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:18.997124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:18.997601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:18.998734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:19.001424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf06a2-48aa-476d-906f-2775d9b22506 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:19.028493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:19.047392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 825/1655] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:19.050691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acab91-3ed7-4561-ad16-f9930e88e247 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:19.056711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:19.058907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:19.059227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:19.059582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] PATH_INFO: `/v3/projects/12ba6569a3614ee0b4213ba2a68261ca/users/919208e7469849fb8c387f892d064bd6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:19.063716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ba6569a3614ee0b4213ba2a68261ca, user_id=919208e7469849fb8c387f892d064bd6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:19.068985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:19.145582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 831/1656] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:19 2022] PUT /identity/v3/projects/12ba6569a3614ee0b4213ba2a68261ca/users/919208e7469849fb8c387f892d064bd6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:19.157974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:19.158260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:19.158539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:19.213389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] MFA Rules not processed for user `919208e7469849fb8c387f892d064bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:19.214680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:19.350477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 826/1657] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:19 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:19.368305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:19.370668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:19.512004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:19.513698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:19.634749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9e5c3-b97c-496a-85fa-42cf40bf6aad admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:19.655399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:19.656858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:19.661340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:19.662053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:19.700254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:19.702277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:19.759963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 832/1658] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:19 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:19.802514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fbdcc-3f7a-402a-a7c2-10639ccc35fb None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:19.816600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:19.819360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:19.821510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:19.822882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] PATH_INFO: `/v3/users/52feac75e3484fe09e2b2af895640995` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:19.830965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52feac75e3484fe09e2b2af895640995)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:19.835102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:19.987469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52feac75e3484fe09e2b2af895640995'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.039953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52feac75e3484fe09e2b2af895640995'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.052715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52feac75e3484fe09e2b2af895640995'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.075295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:20.092405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52feac75e3484fe09e2b2af895640995'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.094194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57b4b765-b813-4472-80e4-b64a1da0a5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52feac75e3484fe09e2b2af895640995'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.130904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:20.145270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 827/1659] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:19 2022] DELETE /identity/v3/users/52feac75e3484fe09e2b2af895640995 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:20.150693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:20.151094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:20.151445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:20.154901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e877-5916-4970-a98b-108943d1b737 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:20.230082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:20.231193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:20.283753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:20.284543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:20.284822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:20.285277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] PATH_INFO: `/v3/users/e8b48e6bcdbf421ab7d53eaaa7f4e36b` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:20.287791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b48e6bcdbf421ab7d53eaaa7f4e36b)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:20.289957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:20.405769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b48e6bcdbf421ab7d53eaaa7f4e36b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.406588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b48e6bcdbf421ab7d53eaaa7f4e36b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.410175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:20.439424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b48e6bcdbf421ab7d53eaaa7f4e36b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.443405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 828/1660] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:20.464385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b48e6bcdbf421ab7d53eaaa7f4e36b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.472654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-1ffb7204-e61d-45ab-a3af-c4bcfe0ba25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b48e6bcdbf421ab7d53eaaa7f4e36b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:20.486451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:20.487063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 833/1661] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:20 2022] DELETE /identity/v3/users/e8b48e6bcdbf421ab7d53eaaa7f4e36b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:20.489168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:20.498430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:20.501627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:20.502413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:20.558485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:20.560126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:20.634655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e1a17-dd37-4042-8cc8-301157ce9455 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:20.728506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 834/1662] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:20.788057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:20.789081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:20.789588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:20.790081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:20.793972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:20.820268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:20.825923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:20.827466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:20.835927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34c37c-eee9-468a-b953-50c02f5e76cf service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:20.847119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfbad1-8f04-4bd7-97ae-d3d9c544ac35 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:20.862326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:20.865461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:20.866339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:20.868777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:20.872163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 829/1663] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:20.898721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:20.902021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dfeeb-dfae-4c65-a152-acd86e80df16 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:20.958017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 835/1664] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:22.012805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:22.013692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:22.014202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:22.081271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] MFA Rules not processed for user `b3a4fdcc0dde49009f4e55d25af4da7c`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:22.082947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.158622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:22.233950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 830/1665] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:22.250829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:22.251822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:22.252250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:22.314096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] MFA Rules not processed for user `b3a4fdcc0dde49009f4e55d25af4da7c`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:22.316994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.348819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 836/1666] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:22.359960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:22.362077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.438349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:22.440087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.474298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0b94-0fbe-4921-a51c-5da13c7383d2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:22.494757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7d963-e773-46bc-b4b9-8edcef6b7416 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:22.500968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:22.503508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:22.503988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:22.504550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] PATH_INFO: `/v3/projects/3be5d64be552499d888f5d6bc6ad8414` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:22.506809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:22.507600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:22.508003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:22.508364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be5d64be552499d888f5d6bc6ad8414)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:22.508502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:22.512749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:22.536935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:22.538911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:22.589656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 831/1667] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:22.714378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be5d64be552499d888f5d6bc6ad8414', 'request_id': 'req-896145dd-3a88-4194-95a1-80a4d8c57a01'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:22.725905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be5d64be552499d888f5d6bc6ad8414', 'request_id': 'req-896145dd-3a88-4194-95a1-80a4d8c57a01'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:22.745029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be5d64be552499d888f5d6bc6ad8414', 'request_id': 'req-896145dd-3a88-4194-95a1-80a4d8c57a01'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:22.754549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:22.756282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.773167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] The token cache is being invalidate because project 3be5d64be552499d888f5d6bc6ad8414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:22.774526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:22.778333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-896145dd-3a88-4194-95a1-80a4d8c57a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:22.789424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 837/1668] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:22 2022] DELETE /identity/v3/projects/3be5d64be552499d888f5d6bc6ad8414 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:22.789902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9481-797f-4b15-bf28-1f24a265ecf2 service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:22.793883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba3ee0-b617-4f9f-8854-3edf138f51e0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:22.794442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba3ee0-b617-4f9f-8854-3edf138f51e0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:22.795068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba3ee0-b617-4f9f-8854-3edf138f51e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:22.831259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-0bba3ee0-b617-4f9f-8854-3edf138f51e0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:22.833521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0bba3ee0-b617-4f9f-8854-3edf138f51e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:22.920932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:23.017584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:23.018383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:23.018738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:23.019055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] PATH_INFO: `/v3/projects/308481ff3a9b480eab9fed402f5224aa` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:23.021199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] RBAC: Authorizing `identity:delete_project(project_id=308481ff3a9b480eab9fed402f5224aa)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:23.024645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:23.064269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 838/1669] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:23.065794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:23.066122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:23.066415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:23.118129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:23.122040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:23.142927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308481ff3a9b480eab9fed402f5224aa', 'request_id': 'req-9e94ae83-d9bc-44d7-b04c-d85ac215765e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.155608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308481ff3a9b480eab9fed402f5224aa', 'request_id': 'req-9e94ae83-d9bc-44d7-b04c-d85ac215765e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.173607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308481ff3a9b480eab9fed402f5224aa', 'request_id': 'req-9e94ae83-d9bc-44d7-b04c-d85ac215765e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.175320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:23.199996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] The token cache is being invalidate because project 308481ff3a9b480eab9fed402f5224aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:23.200283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.202683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.213967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 832/1670] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:22 2022] DELETE /identity/v3/projects/308481ff3a9b480eab9fed402f5224aa => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:23.215704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:23.218194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:23.331663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 839/1671] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:23.332651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:23.362719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e3d12-e457-43c3-af3d-a47a9e602a72 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:23.380292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:23.381268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:23.381557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:23.381905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:23.383708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:23.384401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:23.385871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:23.407342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ae83-d9bc-44d7-b04c-d85ac215765e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:23.418389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:23.419198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:23.419547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:23.419825 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] PATH_INFO: `/v3/users/919208e7469849fb8c387f892d064bd6` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:23.424441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919208e7469849fb8c387f892d064bd6)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:23.426348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:23.452211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 840/1672] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:23.455296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:23.482572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a453a-59ef-475b-905d-f2f6bbacdb6c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:23.496882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:23.496882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:23.518256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:23.518581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:23.520416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:23.525462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:23.590567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919208e7469849fb8c387f892d064bd6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.603170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 841/1673] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:23 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:23.607728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:23.613692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919208e7469849fb8c387f892d064bd6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.616012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919208e7469849fb8c387f892d064bd6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.654198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919208e7469849fb8c387f892d064bd6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.655135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919208e7469849fb8c387f892d064bd6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:23.706712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 833/1674] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:23 2022] DELETE /identity/v3/users/919208e7469849fb8c387f892d064bd6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:23.779959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab1c78-d668-46a0-8d24-63421dff3e97 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:23.915366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:24.009005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:24.010882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:24.011137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:24.014885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:24.017290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:24.024138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:24.130991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 842/1675] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:23 2022] POST /identity/v3/users => generated 285 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:24.132467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:24.134318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:24.173418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919116bf-8b24-40ef-ace8-8158a81cb283 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:24.210765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:24.213914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:24.214211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:24.214522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:24.215234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:24.216904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:24.307753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 834/1676] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:24.312095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:24.328733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32be52c-11f9-4033-8748-200732026977 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:24.366198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:24.366847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:24.367144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:24.367417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:24.368143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:24.369870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb394cf-b796-4bb5-92c4-fcc3f714b3ed admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:24.452750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 843/1677] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:24.454327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:24.473010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5539b-6d33-40b9-b15d-40476d6c8781 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:24.503648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:24.505111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:24.505434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:24.505722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] PATH_INFO: `/v3/projects/be116b8118b84032b5b18ef95cfc5da9/users/40b39847f7344930b0c3962812901b02/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:24.567870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=be116b8118b84032b5b18ef95cfc5da9, user_id=40b39847f7344930b0c3962812901b02, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:24.571127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:24.649446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 835/1678] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:24 2022] PUT /identity/v3/projects/be116b8118b84032b5b18ef95cfc5da9/users/40b39847f7344930b0c3962812901b02/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:24.651259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:24.651695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:24.652316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:24.791305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] MFA Rules not processed for user `40b39847f7344930b0c3962812901b02`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:24.792995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:24.952303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:24.955107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:25.114111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 844/1679] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:25.115060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:25.149758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:25.309178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca39616-5398-4a31-9a60-eb1d540980dc admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:25.321381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:25.322332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.322753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.323136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] PATH_INFO: `/v3/users/5b65afab1e5e44cbbb53fc99a4590178` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.326596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b65afab1e5e44cbbb53fc99a4590178)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:25.336076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:25.436478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d04b7-54e5-4ecf-8943-5ac25221f7fa None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:25.462490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:25.463568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.467842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.469106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.471374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:25.477355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3769963-2597-4caf-9084-58b17066a3e4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:25.487765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b65afab1e5e44cbbb53fc99a4590178'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:25.513684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b65afab1e5e44cbbb53fc99a4590178'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:25.516387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b65afab1e5e44cbbb53fc99a4590178'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:25.562641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b65afab1e5e44cbbb53fc99a4590178'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:25.564879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b65afab1e5e44cbbb53fc99a4590178'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:25.584094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 845/1680] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:25 2022] POST /identity/v3/projects => generated 369 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:25.591744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abcad-d0c0-413a-ab7b-88bc84921d28 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.596986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abcad-d0c0-413a-ab7b-88bc84921d28 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.597285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abcad-d0c0-413a-ab7b-88bc84921d28 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.608269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 846/1681] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 29 12:50:25 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:25.610107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.620229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.621224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.630468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 836/1682] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:24 2022] DELETE /identity/v3/users/5b65afab1e5e44cbbb53fc99a4590178 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:25.631642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:25.634656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:25.739549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] MFA Rules not processed for user `1dd24e8c0d214a0bb62578f9834a40a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:25.743853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:25.817847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:25.872542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893fcf-6bf0-4696-a4f9-0a77cfb2dfd7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:25.887118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:25.887890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.888171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.888447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.889116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:25.898432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414524-6f10-42ed-b573-9f3738ed2d2e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:25.914200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 847/1683] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:25.916440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:25.920028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:25.939970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 837/1684] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:25 2022] POST /identity/v3/users => generated 291 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:25.944568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af99f4-f994-4e45-bed8-87c5848d310f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.944866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af99f4-f994-4e45-bed8-87c5848d310f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.945134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af99f4-f994-4e45-bed8-87c5848d310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.959098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cb3d9-0ec0-44e6-b1ba-602efd77db53 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:25.976280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:25.977593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:25.977883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:25.978390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] PATH_INFO: `/v3/projects/12ba6569a3614ee0b4213ba2a68261ca` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:25.979944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ba6569a3614ee0b4213ba2a68261ca)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:25.985761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:26.074009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-81af99f4-f994-4e45-bed8-87c5848d310f None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:26.076374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-81af99f4-f994-4e45-bed8-87c5848d310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.146265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:26.166252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ba6569a3614ee0b4213ba2a68261ca', 'request_id': 'req-b9119198-6907-4495-9ced-68f651d6d932'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:26.191698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ba6569a3614ee0b4213ba2a68261ca', 'request_id': 'req-b9119198-6907-4495-9ced-68f651d6d932'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:26.215946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ba6569a3614ee0b4213ba2a68261ca', 'request_id': 'req-b9119198-6907-4495-9ced-68f651d6d932'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:26.231890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 838/1685] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:50:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:26.235172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.235632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.236064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.262136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] The token cache is being invalidate because project 12ba6569a3614ee0b4213ba2a68261ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:26.262360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:26.263457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9119198-6907-4495-9ced-68f651d6d932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:26.270870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:26.273278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 848/1686] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:25 2022] DELETE /identity/v3/projects/12ba6569a3614ee0b4213ba2a68261ca => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:26.274603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.274778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9602ac-ad84-44a2-808b-1eccd27abe3c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.277100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9602ac-ad84-44a2-808b-1eccd27abe3c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.277197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9602ac-ad84-44a2-808b-1eccd27abe3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.338039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3e9602ac-ad84-44a2-808b-1eccd27abe3c None None] MFA Rules not processed for user `1dd24e8c0d214a0bb62578f9834a40a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:26.339744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e9602ac-ad84-44a2-808b-1eccd27abe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.433484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 839/1687] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:26.435304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:26.456381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.527534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b5161-8a50-40f1-82cf-51cf594c2a84 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:26.533191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 849/1688] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:26.535465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.536716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.537168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.572793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:26.573632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.574162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.574599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.575456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:26.577293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b4b3c-682b-4517-87e1-4fdf915b4118 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:26.635803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 840/1689] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:26.637240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.637535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.637916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.638864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] MFA Rules not processed for user `b4d57a4e83ab4310b5b3ba90f43c5ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:26.649375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.677043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:26.678526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.691296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 841/1690] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:26.692940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:26.697082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.793104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:26.860703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4b904-4b4a-4098-ab5b-aaca24adef21 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:26.864548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:26.865182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.865467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.865741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.868559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.881475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:26.882970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:26.897177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 850/1691] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:26.897177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:26.897177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.898179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 842/1692] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:26.899114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:26.900727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:26.949368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8072cd6-9d31-4be7-b5d1-b5a5a791b356 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:26.963881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:26.964535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:26.964821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:26.965118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:26.987520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:26.988759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:26.991647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:26.998880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358d6f8-8242-4688-ae19-a76845fdbecf service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.006755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.007397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.007680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.009989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.012085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:27.020151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 851/1693] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:27.021013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.034681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.036322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787f57a-8a81-4752-903f-14df59fa9978 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.043219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7180a7-ea72-4588-a384-22a5a4d91889 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.048737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 843/1694] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:27.049976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.050268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.050642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.056806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.058147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.058147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.058313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] PATH_INFO: `/v3/projects/de7b67dca52145cfbe6f643e999a6017/users/1fb3dba9d1e84e4ba7c70487d0bf62b6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.074865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:27.076282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:27.087442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 844/1695] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:27.090726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7b67dca52145cfbe6f643e999a6017, user_id=1fb3dba9d1e84e4ba7c70487d0bf62b6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.091205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.099883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.120311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc2e5-8e07-47fc-87c7-f32d7983336c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.144308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.145124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.155430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 852/1696] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:27 2022] PUT /identity/v3/projects/de7b67dca52145cfbe6f643e999a6017/users/1fb3dba9d1e84e4ba7c70487d0bf62b6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:27.156802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.157559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.159402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.166079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:27.166547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.169278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.197318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 845/1697] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:50:27.199185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.224285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02614ab-0a6c-4df4-9a8c-45fe60ad2cc9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.358596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674a1af-12fa-427e-8aa9-ec357c108a55 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.371445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.375288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.376118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.376529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.378586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.383603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.410412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 853/1698] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:27.411585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.428367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.429064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.429352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.431630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.432351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.433786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6b7c0-d166-442e-902c-dc38e3508f6d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.434527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.447681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.449238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.449600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.449992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] PATH_INFO: `/v3/projects/de7b67dca52145cfbe6f643e999a6017/users/1fb3dba9d1e84e4ba7c70487d0bf62b6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.454683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7b67dca52145cfbe6f643e999a6017, user_id=1fb3dba9d1e84e4ba7c70487d0bf62b6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.458920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dc3d4-c6dc-4175-aecb-a971c3010e92 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.495185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 854/1699] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:27 2022] PUT /identity/v3/projects/de7b67dca52145cfbe6f643e999a6017/users/1fb3dba9d1e84e4ba7c70487d0bf62b6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:27.497085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.497435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.497786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.517917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 846/1700] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:27 2022] POST /identity/v3/projects => generated 387 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:27.520846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.590091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] MFA Rules not processed for user `1fb3dba9d1e84e4ba7c70487d0bf62b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:27.591339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:27.697059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d474d0-7214-4232-8beb-71c35231a667 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.713901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.715291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.715683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.716212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.717246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.721057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078787c0-9471-4992-8aa7-522397451d23 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.776326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 847/1701] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:27 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:27.778267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.778664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.779043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.808422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 855/1702] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:27.822423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.876274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] MFA Rules not processed for user `e7ee2c3480f8483bb76ecc686215720b`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:27.876788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b7401-e988-4de9-b36f-10a22d32ae38 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:27.879933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:27.897075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:27.898436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:27.898925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:27.899408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:27.900364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:27.902270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:27.917672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 856/1703] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:27.918920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:27.927607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:27.948016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:27.999996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:28.074857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 848/1704] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:28.076209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:28.078126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.094611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e46b-0d0a-42d1-ae10-5e3d54558a75 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.100043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.101210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.101962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.102253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.127392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.129163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.149880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 857/1705] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:28.153082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:28.153586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230ec6-a5ce-4939-ad7b-af111ffffdfd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.171410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f10ea-ceb1-466f-866a-c3fe443d0cc0 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.183997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.184676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.185738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.186277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.186595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.186778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.188927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.192088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.192579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] PATH_INFO: `/v3/projects/dc5920aa5d9c4b8393148fc8ae6db786` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.195331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.197345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5920aa5d9c4b8393148fc8ae6db786)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.201526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.230293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 858/1706] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:28.230293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:28.237968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.408356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5920aa5d9c4b8393148fc8ae6db786', 'request_id': 'req-c274c133-ea32-46a2-89b4-38f67f3bf5ae'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:28.431073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5920aa5d9c4b8393148fc8ae6db786', 'request_id': 'req-c274c133-ea32-46a2-89b4-38f67f3bf5ae'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:28.447792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5920aa5d9c4b8393148fc8ae6db786', 'request_id': 'req-c274c133-ea32-46a2-89b4-38f67f3bf5ae'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:28.450357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f344a81-34f3-42e3-add3-8b4b12ab0aa1 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.499778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] The token cache is being invalidate because project dc5920aa5d9c4b8393148fc8ae6db786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:28.500074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:28.504443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:28.507377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 849/1707] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:28 2022] DELETE /identity/v3/projects/dc5920aa5d9c4b8393148fc8ae6db786 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:28.507995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:28.511556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.612459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.614331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.616884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.617447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.623541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274c133-ea32-46a2-89b4-38f67f3bf5ae admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.625364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.635300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.639757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.641199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.642982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] PATH_INFO: `/v3/projects/3d1a7056497842b49230aa23932fab9a/users/3f09815fd9d04eeeab786d91712bcf82/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.672556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1a7056497842b49230aa23932fab9a, user_id=3f09815fd9d04eeeab786d91712bcf82, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.675720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.708658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 850/1708] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:28 2022] PUT /identity/v3/projects/3d1a7056497842b49230aa23932fab9a/users/3f09815fd9d04eeeab786d91712bcf82/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:28.709779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:28.713215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.762046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:28.771279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:28.799417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1710-2620-483a-b3b3-c6763433350a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:28.805707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:28.806462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.807571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.808121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.810266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:28.842421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.853973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074190c9-8686-4007-a42d-7cb56d7d0a70 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.872199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 859/1709] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:28.884414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969d73-8495-48e4-be5e-124d5ef32cc2 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.884708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969d73-8495-48e4-be5e-124d5ef32cc2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.884989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969d73-8495-48e4-be5e-124d5ef32cc2 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.886395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 860/1710] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 29 12:50:28 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:28.887431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4f83-85e0-44f4-9dae-9142f3517348 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.887793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4f83-85e0-44f4-9dae-9142f3517348 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.887993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4f83-85e0-44f4-9dae-9142f3517348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.926804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:28.928562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3fd3-4c1c-41aa-9f17-401322f8b484 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:28.943482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 851/1711] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:28.945225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bb5a-49b4-4c89-92b8-cb39281d5342 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.945673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bb5a-49b4-4c89-92b8-cb39281d5342 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.946085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bb5a-49b4-4c89-92b8-cb39281d5342 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.948518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 852/1712] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 29 12:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:28.949719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ada8d-6200-4d6a-8b72-87afab6d6074 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:28.950477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ada8d-6200-4d6a-8b72-87afab6d6074 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:28.951130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ada8d-6200-4d6a-8b72-87afab6d6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:28.955173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a71b4f83-85e0-44f4-9dae-9142f3517348 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:28.961498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a71b4f83-85e0-44f4-9dae-9142f3517348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.024003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-208ada8d-6200-4d6a-8b72-87afab6d6074 None None] MFA Rules not processed for user `3f09815fd9d04eeeab786d91712bcf82`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:29.029876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-208ada8d-6200-4d6a-8b72-87afab6d6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.157760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 861/1713] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:29.157760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.157760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.157760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.222607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:29.223869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.307019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 853/1714] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:29.308605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216f75-372a-499a-bf47-fa4d932c2694 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.308940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216f75-372a-499a-bf47-fa4d932c2694 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.309376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216f75-372a-499a-bf47-fa4d932c2694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.365375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 862/1715] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 29 12:50:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:29.366878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:29.370982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.380250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f8216f75-372a-499a-bf47-fa4d932c2694 None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:29.382812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f8216f75-372a-499a-bf47-fa4d932c2694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.531534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:29.536574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 854/1716] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:50:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:29.539804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.540361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.540776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.544721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f60089-5d9d-403e-aa0c-b3278a306f87 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:29.565696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:29.566313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.566592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.566867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.570333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.581183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:29.584142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.608542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 855/1717] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:29.610057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:29.649058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c2e82-cdde-405c-b418-c0e8a580aa44 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:29.662188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:29.663573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.665176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.665528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.668575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:29.676072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:29.748334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 856/1718] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:29 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:29.752399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:29.762083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:29.765056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:29.765436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:29.792243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 863/1719] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:29.793574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:29.795390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.824037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ecef-9f74-42be-897f-e3086ef6f804 service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:29.881118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e132cd-0279-4caf-946a-b92248ee3426 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:29.888604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:29.890371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.890708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.891134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:29.893191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:29.912648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:29.914978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:29.931118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 857/1720] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:29.932127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:29.979701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:29.983053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:29.984840 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:29.985142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.011665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.013104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.013664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb407c-c3ef-4dcf-8ff6-e824bc49f918 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.019802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.020469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.020883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.021163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.021969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:30.022278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.024197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.034427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 864/1721] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:30.035236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.038400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 858/1722] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:30.042803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.067529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47dc97-9acc-4512-b640-0de9d53b7855 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.068303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bbbe6-b984-4c91-ad1b-b958e901b48e service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.080230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.081564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.081688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.081877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.082162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.082297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.082575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.082848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.082964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.083514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.085158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.087394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.156218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 859/1723] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:30 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:30.161689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.184730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 865/1724] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:30 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:30.186385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.213495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df49d-3150-4b16-a787-43a37173521a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.275854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f8ba-90d5-4c6f-8927-afd48247455d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.288084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.289896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.290198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.291019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.291642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.300702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.346840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:30.375913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 860/1725] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:30 2022] POST /identity/v3/users => generated 282 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:30.377621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.413388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8c96-783b-4e9c-9b04-c414f2e2ee9e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.428859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.437282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.437626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.437927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.441038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.442168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.442285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.442285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.442782 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.443509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.444915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.447134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.478981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 866/1726] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:30.484432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 861/1727] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:30.487358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.487827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.515739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7a2a-ec94-4915-8413-8a854b0ff82b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.531593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408e0aa-d108-4aee-b24a-688801f5d228 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.540345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.542790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.543228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.544483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.545432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.550921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.581161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.581943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.582405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.584659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] PATH_INFO: `/v3/projects/fee19c20545341b29bc8e2b006e62fc9/users/8da40de908a142f98c12b3e6b648657e/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.598368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 862/1728] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:30.599361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.622191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541875c-b7e0-42c8-87f1-c253eaa4a67f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:30.628553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:30.629344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.634423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.634718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] PATH_INFO: `/v3/projects/5e0673cad8e1418a92407e2b9fa55a8c/users/3cc3d87c8d8144e89699dd99ef61bb96/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.685488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0673cad8e1418a92407e2b9fa55a8c, user_id=3cc3d87c8d8144e89699dd99ef61bb96, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.699054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1b4ca-be98-4ecb-ab06-87b19408d9b5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.712506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee19c20545341b29bc8e2b006e62fc9, user_id=8da40de908a142f98c12b3e6b648657e, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:30.729863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:30.751180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 863/1729] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:30 2022] PUT /identity/v3/projects/5e0673cad8e1418a92407e2b9fa55a8c/users/3cc3d87c8d8144e89699dd99ef61bb96/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:30.755852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:30.758890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:30.759767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:30.776481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:30.788221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 867/1730] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:30 2022] PUT /identity/v3/projects/fee19c20545341b29bc8e2b006e62fc9/users/8da40de908a142f98c12b3e6b648657e/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:30.799003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:30.914487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] MFA Rules not processed for user `3cc3d87c8d8144e89699dd99ef61bb96`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:30.915721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:31.048533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59551-2983-4172-a87e-ccaa7bb82e0e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:31.061464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:31.063673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.063980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.065933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.066748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:31.068746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:31.134484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 868/1731] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:31.137276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:31.151368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 864/1732] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:31.156267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:31.164295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43160774-c275-4ed4-994e-1b02e127d40e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:31.202429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:31.203277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.203561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.203841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] PATH_INFO: `/v3/projects/fee19c20545341b29bc8e2b006e62fc9/users/8da40de908a142f98c12b3e6b648657e/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.208368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f594ec-c92b-41da-b4ca-cfce7f810278 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:31.210401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee19c20545341b29bc8e2b006e62fc9, user_id=8da40de908a142f98c12b3e6b648657e, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:31.214506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aace6b-851b-4f9f-9258-fc308deb3d58 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:31.222629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:31.223268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.223546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.223815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.224485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:31.233323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:31.274142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 869/1733] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:31 2022] PUT /identity/v3/projects/fee19c20545341b29bc8e2b006e62fc9/users/8da40de908a142f98c12b3e6b648657e/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:31.275921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ad1-4a3e-4c93-8ac4-93803176eef7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.276208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ad1-4a3e-4c93-8ac4-93803176eef7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.276481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386ad1-4a3e-4c93-8ac4-93803176eef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.364405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 865/1734] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:31 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:31.369138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:31.443453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-37386ad1-4a3e-4c93-8ac4-93803176eef7 None None] MFA Rules not processed for user `8da40de908a142f98c12b3e6b648657e`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:31.446014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-37386ad1-4a3e-4c93-8ac4-93803176eef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:31.754206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f4a3-5447-4aae-bf28-a05f8784a222 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:31.782700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:31.783347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.783639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.783910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.784584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:31.795408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:31.838229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 870/1735] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 563 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:31.846198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:31.847710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:31.848245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:31.857083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 866/1736] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:31 2022] POST /identity/v3/users => generated 289 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:31.858005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:31.872116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:31.896024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] MFA Rules not processed for user `1dd24e8c0d214a0bb62578f9834a40a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:31.899180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:31.924784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:32.057346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 871/1737] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:32.057346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:32.100263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809ed62-d8b8-491a-bd7a-ce3a2a966423 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:32.123454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:32.126229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.127851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.128339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.129254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:32.133358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:32.163569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee7ba-b6e8-433d-b309-e36eef7d8941 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:32.168588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 872/1738] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:32.172059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:32.183010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:32.194842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6c54-2393-4c10-8b9f-3942f431807c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:32.196661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:32.205341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:32.205499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.205693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.205997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.206101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.206362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.206655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] PATH_INFO: `/v3/projects/3612f0db272941f7a6b7ea0c950a6097/users/b0b8a00be5554bd4aa83f43d310a5df9/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.246776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3612f0db272941f7a6b7ea0c950a6097, user_id=b0b8a00be5554bd4aa83f43d310a5df9, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:32.266178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:32.268267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:32.269172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:32.308169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 867/1739] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:32.340136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 873/1740] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:32 2022] PUT /identity/v3/projects/3612f0db272941f7a6b7ea0c950a6097/users/b0b8a00be5554bd4aa83f43d310a5df9/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:32.344206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:32.360014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0665e-969a-476c-b0ea-1a6ae6f8b8dd service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:32.610865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:32.612682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.613114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.614192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.616598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:32.638206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01b8e4-5873-4170-ac62-fa840b0412d8 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:32.689316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 868/1741] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:32.690760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:32.707290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8fdc9-243a-40ac-81cd-5801425d8746 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:32.736737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:32.737510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.739480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.740429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] PATH_INFO: `/v3/projects/3612f0db272941f7a6b7ea0c950a6097/users/b0b8a00be5554bd4aa83f43d310a5df9/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.744190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3612f0db272941f7a6b7ea0c950a6097, user_id=b0b8a00be5554bd4aa83f43d310a5df9, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:32.748474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:32.788237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 874/1742] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:32 2022] PUT /identity/v3/projects/3612f0db272941f7a6b7ea0c950a6097/users/b0b8a00be5554bd4aa83f43d310a5df9/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:32.798723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:32.799260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:32.799260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:32.912019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] MFA Rules not processed for user `b0b8a00be5554bd4aa83f43d310a5df9`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:32.915629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:33.144244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 869/1743] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:33.305784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:33.532904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232466-176d-41c5-84d5-da6b1f9e4482 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:33.544787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:33.545448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:33.545795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:33.546196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:33.594230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:33.596200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98faf08-9c31-47cc-93a7-df8e1d3d9749 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:33.649907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 875/1744] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:33.978808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:34.010480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85796779-029e-4c2e-a365-97b8e5f0ec30 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:34.018075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:34.018933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:34.019486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:34.019935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:34.109552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:34.198723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:34.199476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:34.199976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:34.231451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:34.236535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18482e3a-2a8e-4648-beea-0b21fb2825cf service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:34.301636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 870/1745] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:34.346357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:34.348872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:34.478311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 876/1746] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 29 12:50:34.681906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:34.682617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:34.683945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:34.717383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:34.768493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] MFA Rules not processed for user `b4d57a4e83ab4310b5b3ba90f43c5ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:34.773139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:34.856154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:34.916500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732c5e2-0381-4c62-a07c-575c694c8a5e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:34.925189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:34.926311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:34.926756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:34.927167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] PATH_INFO: `/v3/users/3f09815fd9d04eeeab786d91712bcf82` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:34.938798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f09815fd9d04eeeab786d91712bcf82)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:34.943094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:35.013843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 871/1747] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:35.018721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:35.024559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:35.034479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f09815fd9d04eeeab786d91712bcf82'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.035471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f09815fd9d04eeeab786d91712bcf82'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.055618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f09815fd9d04eeeab786d91712bcf82'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.090009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f09815fd9d04eeeab786d91712bcf82'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.110092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-32c79beb-0643-4c47-9e4c-8fa7b3387a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f09815fd9d04eeeab786d91712bcf82'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.161415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 877/1748] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:34 2022] DELETE /identity/v3/users/3f09815fd9d04eeeab786d91712bcf82 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:35.163432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:35.163748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:35.164137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:35.257504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:35.264097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:35.290286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223306d6-4eb3-45f8-8bda-84798107de74 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:35.461119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:35.499569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:35.500997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:35.504615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:35.506324 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:35.516335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:35.531483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 878/1749] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:35.626086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:35.628019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:35.654282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e4ab4-8e09-45c9-9a41-53c0ae76ec2a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:35.671855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:35.674821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:35.680562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:35.684201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:35.684482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:35.684765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] PATH_INFO: `/v3/users/40b39847f7344930b0c3962812901b02` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:35.694500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b39847f7344930b0c3962812901b02)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:35.698809 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:35.722688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 872/1750] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:50:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:50:35.723051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:35.724879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:35.901063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739a6d-e1f1-4017-9d84-08c795c1ef1d service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:35.908396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:35.909326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:35.911602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:35.912204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:35.928984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:35.931480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf1ce1-3220-4232-85de-bde0b2e98ef8 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:35.947145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b39847f7344930b0c3962812901b02'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.947907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b39847f7344930b0c3962812901b02'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.951230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 873/1751] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:35.975884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b39847f7344930b0c3962812901b02'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:35.978270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:36.001719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b39847f7344930b0c3962812901b02'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:36.016034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b39847f7344930b0c3962812901b02'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:36.027289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 879/1752] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:35 2022] DELETE /identity/v3/users/40b39847f7344930b0c3962812901b02 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:36.032194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79938-f1bd-41f0-924a-348043379536 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:36.032731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79938-f1bd-41f0-924a-348043379536 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:36.033259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79938-f1bd-41f0-924a-348043379536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:36.081539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c1a79938-f1bd-41f0-924a-348043379536 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:36.081539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c1a79938-f1bd-41f0-924a-348043379536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:36.266208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 874/1753] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:36.367647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:36.369459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:36.516316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d94557-011e-4096-a8a9-6ec91efcefee admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:36.520742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:36.522473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:36.522939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:36.523311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:36.550441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:36.552548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e936206-3ca7-4996-b9ea-d2e20a8ed054 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:36.575558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 880/1754] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:37.063574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:37.064110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:37.064110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:37.136817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] MFA Rules not processed for user `e7ee2c3480f8483bb76ecc686215720b`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:37.144673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:37.286461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:37.366991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 875/1755] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:37.990311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:37.990837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:37.991082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:38.060063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:38.083905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:38.115935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 881/1756] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Jul 29 12:50:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:38.220579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:38.222505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:38.274777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6f4c4-67d4-41a5-9022-98a7f58c9b70 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:38.306531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:38.307418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:38.308384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:38.309966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] PATH_INFO: `/v3/projects/be116b8118b84032b5b18ef95cfc5da9` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:38.314987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be116b8118b84032b5b18ef95cfc5da9)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:38.318456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:38.372236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:38.373794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:38.405724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94468635-3f3a-4bc5-a541-2d566526f2a2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:38.434663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:38.435750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:38.436250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:38.436944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:38.469943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:38.477798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:38.523124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 882/1757] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:38.584798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be116b8118b84032b5b18ef95cfc5da9', 'request_id': 'req-a6dde38c-322c-4e62-aae8-7803953c0693'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:38.605465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be116b8118b84032b5b18ef95cfc5da9', 'request_id': 'req-a6dde38c-322c-4e62-aae8-7803953c0693'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:38.619874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be116b8118b84032b5b18ef95cfc5da9', 'request_id': 'req-a6dde38c-322c-4e62-aae8-7803953c0693'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:38.650399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] The token cache is being invalidate because project be116b8118b84032b5b18ef95cfc5da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:38.652775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:38.658750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:38.667114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 876/1758] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:38 2022] DELETE /identity/v3/projects/be116b8118b84032b5b18ef95cfc5da9 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:38.671320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:38.699765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac042e-a129-456a-8890-5a12e59d25ca service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:38.905243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:38.906844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:38.907342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:38.907759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] PATH_INFO: `/v3/users/1fb3dba9d1e84e4ba7c70487d0bf62b6` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:38.910844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb3dba9d1e84e4ba7c70487d0bf62b6)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:38.917351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:39.047708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb3dba9d1e84e4ba7c70487d0bf62b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:39.049127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb3dba9d1e84e4ba7c70487d0bf62b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:39.064273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb3dba9d1e84e4ba7c70487d0bf62b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:39.068021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:39.091416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb3dba9d1e84e4ba7c70487d0bf62b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:39.125043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb3dba9d1e84e4ba7c70487d0bf62b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:39.142646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 883/1759] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:38 2022] DELETE /identity/v3/users/1fb3dba9d1e84e4ba7c70487d0bf62b6 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:39.978604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:39.980083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:39.997492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde38c-322c-4e62-aae8-7803953c0693 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:40.115665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:40.116738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:40.117268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:40.117759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] PATH_INFO: `/v3/projects/3d1a7056497842b49230aa23932fab9a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:40.119696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1a7056497842b49230aa23932fab9a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:40.122841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:40.283863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1a7056497842b49230aa23932fab9a', 'request_id': 'req-80519397-1013-48af-9d0e-8b47682a3d0d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.304614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1a7056497842b49230aa23932fab9a', 'request_id': 'req-80519397-1013-48af-9d0e-8b47682a3d0d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.323123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1a7056497842b49230aa23932fab9a', 'request_id': 'req-80519397-1013-48af-9d0e-8b47682a3d0d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.343631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:40.351787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] The token cache is being invalidate because project 3d1a7056497842b49230aa23932fab9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:40.357442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.358873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.361547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 877/1760] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:39 2022] DELETE /identity/v3/projects/3d1a7056497842b49230aa23932fab9a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:40.365417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:40.367140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:40.395372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46e9b4-4bb9-4669-a503-69215077fc2d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:40.525284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:40.526633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:40.528808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:40.529203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] PATH_INFO: `/v3/users/8da40de908a142f98c12b3e6b648657e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:40.531779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da40de908a142f98c12b3e6b648657e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:40.534119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:40.675051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da40de908a142f98c12b3e6b648657e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.676008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da40de908a142f98c12b3e6b648657e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.710630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da40de908a142f98c12b3e6b648657e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.749910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da40de908a142f98c12b3e6b648657e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.828386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2fa10e18-ac88-47ae-bc10-5f3b842d0fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da40de908a142f98c12b3e6b648657e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:40.858805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 884/1761] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:40 2022] DELETE /identity/v3/users/8da40de908a142f98c12b3e6b648657e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:41.072694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:41.074237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:41.103771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80519397-1013-48af-9d0e-8b47682a3d0d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:41.284301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:41.284975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:41.285255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:41.286262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] PATH_INFO: `/v3/projects/de7b67dca52145cfbe6f643e999a6017` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:41.287920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7b67dca52145cfbe6f643e999a6017)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:41.290033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:41.470683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7b67dca52145cfbe6f643e999a6017', 'request_id': 'req-f4726425-c7f9-4a78-89a0-64c43011d7a4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:41.485143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7b67dca52145cfbe6f643e999a6017', 'request_id': 'req-f4726425-c7f9-4a78-89a0-64c43011d7a4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:41.509454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7b67dca52145cfbe6f643e999a6017', 'request_id': 'req-f4726425-c7f9-4a78-89a0-64c43011d7a4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:41.538876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] The token cache is being invalidate because project de7b67dca52145cfbe6f643e999a6017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:41.539452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:41.541583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f4726425-c7f9-4a78-89a0-64c43011d7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:41.555869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 878/1762] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:41 2022] DELETE /identity/v3/projects/de7b67dca52145cfbe6f643e999a6017 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:41.557760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:41.558747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:41.559250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:41.608972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:41.613420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:41.697232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:41.721659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 885/1763] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:41.732237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:41.732758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:41.733186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:41.763435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:41.765383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:41.780918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 879/1764] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:41.785672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:41.806056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f4b8-077c-4be7-8379-6e7a7f77b58a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:41.812995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:41.813977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:41.814994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:41.815435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:41.816307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:41.817223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:41.819498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:41.901957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 886/1765] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:50:41.903926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:41.941532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d83e6f-bb47-4d9f-99cb-633d74e08ae6 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:41.957770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:41.959769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:41.960254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:41.960758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:41.963231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:41.968052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.035657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 880/1766] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:41 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:42.039715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:42.097813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19407997-cf90-49fb-8cd9-4b4996644900 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:42.338202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:42.338936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.339223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.339502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.340313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:42.342455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.430205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 887/1767] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:42 2022] POST /identity/v3/users => generated 297 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:42.431606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:42.464412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c66e-c305-4223-a1c6-abd756c354ec admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:42.483465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:42.485445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.495975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.496614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.497700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:42.503406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.552620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 881/1768] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:42.557758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:42.573530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef663e8-3eae-45f2-9500-7679567443d4 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:42.598387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:42.599282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.599825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.600278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.601147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:42.603768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.636120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:42.638013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 888/1769] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:42.651183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1707-8b4d-47f5-a7a9-76b6a1d2440a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:42.679703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:42.685460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.686194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.690543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] PATH_INFO: `/v3/projects/ad47d1e9abac472bb18ce63879e9def2/users/af6703e3a63d4846848d275e464b94b8/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.702868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:42.707451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:42.736685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e4acf-ef5e-4486-a347-52e0ec3c1063 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:42.744845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:42.745744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.746378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.746964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] PATH_INFO: `/v3/projects/fee19c20545341b29bc8e2b006e62fc9` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.749714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee19c20545341b29bc8e2b006e62fc9)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:42.750260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad47d1e9abac472bb18ce63879e9def2, user_id=af6703e3a63d4846848d275e464b94b8, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:42.752331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.753193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dafdc4-fde6-420f-bf17-59f9361ec5d1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:42.781790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 882/1770] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:42 2022] PUT /identity/v3/projects/ad47d1e9abac472bb18ce63879e9def2/users/af6703e3a63d4846848d275e464b94b8/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:42.792935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0599a-db40-4ed0-a81e-72a4fdf512c7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.793467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0599a-db40-4ed0-a81e-72a4fdf512c7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.797342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0599a-db40-4ed0-a81e-72a4fdf512c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.878605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-73f0599a-db40-4ed0-a81e-72a4fdf512c7 None None] MFA Rules not processed for user `af6703e3a63d4846848d275e464b94b8`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:42.885468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-73f0599a-db40-4ed0-a81e-72a4fdf512c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:42.910921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee19c20545341b29bc8e2b006e62fc9', 'request_id': 'req-ce62d648-8772-4867-95f3-a2f85cb0df0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:42.930503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:42.930646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee19c20545341b29bc8e2b006e62fc9', 'request_id': 'req-ce62d648-8772-4867-95f3-a2f85cb0df0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:42.950608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee19c20545341b29bc8e2b006e62fc9', 'request_id': 'req-ce62d648-8772-4867-95f3-a2f85cb0df0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:42.976400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] The token cache is being invalidate because project fee19c20545341b29bc8e2b006e62fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:42.977184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:42.986921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce62d648-8772-4867-95f3-a2f85cb0df0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:42.986921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 889/1771] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:42 2022] DELETE /identity/v3/projects/fee19c20545341b29bc8e2b006e62fc9 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:42.987387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bc42-687e-4468-b22f-e55ca33d71e3 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:42.987843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bc42-687e-4468-b22f-e55ca33d71e3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:42.988309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bc42-687e-4468-b22f-e55ca33d71e3 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:42.999605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 890/1772] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:50:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:43.001275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.001735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.002222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.034669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:43.036681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.041377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 883/1773] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:43.041377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.041377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.041377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.063534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:43.064585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.086558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:43.123066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 891/1774] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:43.123737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.128971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.142146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bd1d-5bf8-4332-86ad-34d8de774b11 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.147369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.148017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.149975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.150628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.151367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.153402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df8b9c-9878-47f7-bd1f-bca694383a6b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.190631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 892/1775] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:43.192312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.192868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.193096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.216391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:43.218301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.219983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 884/1776] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:43.221320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.239920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0592ebf-b8d7-4147-8a67-e8f7c8ebc948 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.293345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 893/1777] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:43.294810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.295996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.315016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.315746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.316120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.316465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.317227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.322877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.365650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 885/1778] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/users => generated 303 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:43.365650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.381048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc52a-699f-465b-b3c3-9fe3f0a1d3d4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.389418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.390780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.391073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.391359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.392156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:43.392912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.394980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.414553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 886/1779] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:43.415199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7e8ff-b465-4249-a95b-88d3fe5bb074 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.415317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.422054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.422973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.423429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.423848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.426549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20215d-6427-41af-8730-c9aeaf5607c1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.431591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.432215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.432748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.432993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.433523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.435210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.448265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 887/1780] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:43.449252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.461986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e075a86-157d-47eb-b065-2d0ade78d345 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.463374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.465066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.466962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.467607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.467930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.468206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.469013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.471246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.482287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 894/1781] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:43.483965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.486250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:43.498615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b43e-d43b-49f4-bdf8-f78c03c55540 service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.504166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.504637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 888/1782] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:43.505115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.505684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.506007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] PATH_INFO: `/v3/projects/925ea98a59144d38b4d649895056a76d/users/abc6bcd2c747408296de4e63c669e1ef/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.508772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.527337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925ea98a59144d38b4d649895056a76d, user_id=abc6bcd2c747408296de4e63c669e1ef, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.531496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.565191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 895/1783] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:43 2022] PUT /identity/v3/projects/925ea98a59144d38b4d649895056a76d/users/abc6bcd2c747408296de4e63c669e1ef/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:43.569968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.600710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921f43-858f-4e74-ac8d-2884a11789c2 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.601243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:43.646369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c01d41-75ba-4ef2-8087-98bc320da339 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.652405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.653057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.653483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.653764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.654785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.656692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.668269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.669059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.669413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.669791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.670608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.673891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 896/1784] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:43.674045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.675817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.687595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7dc6f-4edb-4739-a823-6ed48eb6f113 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.692514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.693182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.693463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.693739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] PATH_INFO: `/v3/projects/925ea98a59144d38b4d649895056a76d/users/abc6bcd2c747408296de4e63c669e1ef/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.695735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925ea98a59144d38b4d649895056a76d, user_id=abc6bcd2c747408296de4e63c669e1ef, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.698408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a289-09b5-4222-b75e-95f83083d833 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.700479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 889/1785] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:43.705862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.719540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f12b5-1e68-4879-b858-b67f3076de12 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.721609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 897/1786] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:43 2022] PUT /identity/v3/projects/925ea98a59144d38b4d649895056a76d/users/abc6bcd2c747408296de4e63c669e1ef/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:43.732030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.732549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.732896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.774124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] MFA Rules not processed for user `abc6bcd2c747408296de4e63c669e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:43.775551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.795280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.796086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.796894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.796894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.797632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.799491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.820830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 890/1787] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:43.821851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.834883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce31dc-29e7-4f09-b01f-1cd1d348a6d7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.839661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.840260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.840543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.840819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.841507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.844627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.854787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 891/1788] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:43.860378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.867306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 898/1789] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:43.874472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2b817-f018-4f28-bffc-39e7a6689e04 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.878878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:43.879266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.880245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.880536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.880814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] PATH_INFO: `/v3/projects/00e9a081e6cf4b76bc2a9c9590d784eb/users/dde88060aae14a7893417e19122447cb/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.880986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.893082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:43.897150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e9a081e6cf4b76bc2a9c9590d784eb, user_id=dde88060aae14a7893417e19122447cb, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:43.900881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe09035-63be-4abf-9b6b-3b476f29ccc9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:43.915131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 892/1790] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:43 2022] PUT /identity/v3/projects/00e9a081e6cf4b76bc2a9c9590d784eb/users/dde88060aae14a7893417e19122447cb/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:43.926868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.927158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.927436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:43.940693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f48e42-244e-4086-9879-62fd11dea137 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:43.956281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] MFA Rules not processed for user `dde88060aae14a7893417e19122447cb`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:43.957578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:43.985527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:43.992864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:43.993585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:43.993982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:43.994316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.026915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 893/1791] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:44.032167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.057092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.058619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.070344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 899/1792] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:44.098164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a019903-1cf6-4646-b0e9-fa12e2a5722c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.102853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.103534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.103747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.104023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.104689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.106675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.129924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.131351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:44.137658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 894/1793] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:44 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:44.140249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.143325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc25c38-c174-4823-8ec5-c0b7ee6bf3c3 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.200124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.200754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.201037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.201314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.205877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bdcfa-01d8-4d60-845c-e2a8dccff2e6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.212991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.213606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.213927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.214205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.214876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.216792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.228936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:44.250290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 895/1794] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:44 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:44.251793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.262204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.263690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.264118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec1bd-4e0a-49e5-b75b-5e99fe19232d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.270446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.271069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.271351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.271651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.272360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.274020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.278881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 900/1795] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:44.285439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 896/1796] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:44.289447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.301079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfb2fa-3571-497e-9725-aca40f38b152 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.313613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.314349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.314635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.314917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] PATH_INFO: `/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c/users/b0b2387574a8460bb0207ae85fadaa1c/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.330740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8339fb8cd2467891ac96ab9c5f769c, user_id=b0b2387574a8460bb0207ae85fadaa1c, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.333402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.347758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 901/1797] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:44 2022] PUT /identity/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c/users/b0b2387574a8460bb0207ae85fadaa1c/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:44.351866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.364212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2db70-7f68-441a-878e-36203dd47c91 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.452763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.453564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.454039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.454388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.455185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.457234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d8801-6d86-48cb-8026-bd0d278c65b4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.471336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 897/1798] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:44.475343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.487884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416129-564d-4ace-a4c7-1d730da12e31 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.494133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.495164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.495525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.495856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] PATH_INFO: `/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c/users/b0b2387574a8460bb0207ae85fadaa1c/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.498542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8339fb8cd2467891ac96ab9c5f769c, user_id=b0b2387574a8460bb0207ae85fadaa1c, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.502049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.533870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 902/1799] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:44 2022] PUT /identity/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c/users/b0b2387574a8460bb0207ae85fadaa1c/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:44.537734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.538180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.538462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.591238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] MFA Rules not processed for user `b0b2387574a8460bb0207ae85fadaa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:44.593946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:44.702868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 898/1800] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:44.716119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:44.731505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:44.780837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bb871-bf0d-40fa-b388-e67381f04384 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:44.785083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:44.785648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:44.785966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:44.786243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:44.795725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:44.797978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333f6e1-a1a8-4428-8c0b-6faa3053a9bb service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:44.807712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 903/1801] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:45.240885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:45.303874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80484ef-8cbd-4102-b0bf-c1e24330a368 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:45.308297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:45.308931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:45.309213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:45.309486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] PATH_INFO: `/v3/users/dde88060aae14a7893417e19122447cb` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:45.310707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde88060aae14a7893417e19122447cb)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:45.312712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:45.322646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:45.390644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde88060aae14a7893417e19122447cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:45.398446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde88060aae14a7893417e19122447cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:45.399241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde88060aae14a7893417e19122447cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:45.413345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde88060aae14a7893417e19122447cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:45.417549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde88060aae14a7893417e19122447cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:45.449240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 899/1802] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:45 2022] DELETE /identity/v3/users/dde88060aae14a7893417e19122447cb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:45.454026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:45.454661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:45.455147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:45.480613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:45.482754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:45.632670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 904/1803] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:45.642786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:45.644385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:45.722064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491679-0066-4622-ae79-99ef117ba15d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:45.731071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:45.732093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:45.732576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:45.732865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:45.746756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:45.748205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:45.763845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 900/1804] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:46.113900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:46.113900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:46.128716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82609ef-ea7b-4711-91ec-db07a986ee58 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:46.134750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:46.136092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:46.136602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:46.137134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] PATH_INFO: `/v3/projects/00e9a081e6cf4b76bc2a9c9590d784eb` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:46.139014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e9a081e6cf4b76bc2a9c9590d784eb)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:46.142018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:46.208011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e9a081e6cf4b76bc2a9c9590d784eb', 'request_id': 'req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.214869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e9a081e6cf4b76bc2a9c9590d784eb', 'request_id': 'req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.225713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e9a081e6cf4b76bc2a9c9590d784eb', 'request_id': 'req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.246188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] The token cache is being invalidate because project 00e9a081e6cf4b76bc2a9c9590d784eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:46.246524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.247204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.253340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 905/1805] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:46 2022] DELETE /identity/v3/projects/00e9a081e6cf4b76bc2a9c9590d784eb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:46.253643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:46.276549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807133-22dd-47b0-9042-6cad8ee0c9a2 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:46.368747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:46.369337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:46.369612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:46.369933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] PATH_INFO: `/v3/users/b0b2387574a8460bb0207ae85fadaa1c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:46.371133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b2387574a8460bb0207ae85fadaa1c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:46.373190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:46.406093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:46.424878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b2387574a8460bb0207ae85fadaa1c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.437568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b2387574a8460bb0207ae85fadaa1c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.438428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b2387574a8460bb0207ae85fadaa1c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.450863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b2387574a8460bb0207ae85fadaa1c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.451520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-72fde688-08e5-41f8-8eaa-ccfe564b097f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b2387574a8460bb0207ae85fadaa1c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:46.466929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 901/1806] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:46 2022] DELETE /identity/v3/users/b0b2387574a8460bb0207ae85fadaa1c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:46.884966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:46.886584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:46.913066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6f9b-b52b-43d5-a68d-4d10c33bfdb0 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:46.975135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:46.998242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:46.998927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:46.999212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:46.999489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] PATH_INFO: `/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.000652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8339fb8cd2467891ac96ab9c5f769c)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.002606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.054112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8339fb8cd2467891ac96ab9c5f769c', 'request_id': 'req-9737e117-d62b-4140-96cf-da59d6e31406'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:47.072059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8339fb8cd2467891ac96ab9c5f769c', 'request_id': 'req-9737e117-d62b-4140-96cf-da59d6e31406'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:47.083892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8339fb8cd2467891ac96ab9c5f769c', 'request_id': 'req-9737e117-d62b-4140-96cf-da59d6e31406'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:47.097739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] The token cache is being invalidate because project 2a8339fb8cd2467891ac96ab9c5f769c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:47.098696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:47.098841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9737e117-d62b-4140-96cf-da59d6e31406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:47.100598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 906/1807] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:46 2022] DELETE /identity/v3/projects/2a8339fb8cd2467891ac96ab9c5f769c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:47.105641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.109481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.109903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.138127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:47.142607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:47.230027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 902/1808] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:47.239342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.239882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.239882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.261056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:47.262410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:47.273271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 907/1809] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:47.275646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.286125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c73474-1802-4fa0-87a7-65fde84f999d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.292922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.292922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.293179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.293327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.294069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:47.294377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.296706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.309842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 903/1810] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:47.314480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.324549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cd2b5-822b-4024-9d42-cb36cfa895e0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.333475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.334150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.334433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.334748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.335479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.337401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.386351 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 908/1811] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:47 2022] POST /identity/v3/projects => generated 359 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:47.387598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.401936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a484fc1-a380-4f14-b658-3b0a2250107c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.484297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.485152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.485534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.485793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.486682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.488899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.517852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 904/1812] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:47 2022] POST /identity/v3/users => generated 280 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:47.523466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.536286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d83c-850e-4ab2-a3d8-4965042b1180 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.541182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.541811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.542176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.542451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.543152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.544782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.555802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 909/1813] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:47.559869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.570777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28654f2-f396-4f43-a13f-33eb39eb4c82 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.575329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.575942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.576224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.576536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.577265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.578893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c56245-76bc-4a4a-9fa4-23b8788e0643 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.593727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 905/1814] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:47.595040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.606083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeda99-ed91-48c1-8f95-ea3fb8effb50 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.611713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.612388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.612663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.612958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] PATH_INFO: `/v3/projects/18b8a5cdbf314feab101e309f3b58478/users/19b447eb1e67432186cd0bc3f9963dcc/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.629615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b8a5cdbf314feab101e309f3b58478, user_id=19b447eb1e67432186cd0bc3f9963dcc, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.632367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.648807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 910/1815] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:47 2022] PUT /identity/v3/projects/18b8a5cdbf314feab101e309f3b58478/users/19b447eb1e67432186cd0bc3f9963dcc/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:47.659377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.659591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.659868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.682981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:47.701474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] MFA Rules not processed for user `19b447eb1e67432186cd0bc3f9963dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:47.703353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:47.795358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 906/1816] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:47 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:47.834676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:47.837375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:47.932275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e9ee0-2e65-4ec6-8fb7-3880eda2b990 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:47.939608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:47.940469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:47.942269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:47.942634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:47.957850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:47.957850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36be3bd-7af9-4bac-a269-c3f3a10c4277 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:47.976681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 911/1817] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:47 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:48.000157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:48.097643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafa9f-59f9-4cef-be97-e98f90da9878 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:48.107210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:48.108013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:48.108335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:48.108662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] PATH_INFO: `/v3/users/19b447eb1e67432186cd0bc3f9963dcc` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:48.112765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b447eb1e67432186cd0bc3f9963dcc)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:48.114866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:48.173174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b447eb1e67432186cd0bc3f9963dcc'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.181365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b447eb1e67432186cd0bc3f9963dcc'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.182383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b447eb1e67432186cd0bc3f9963dcc'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.194948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b447eb1e67432186cd0bc3f9963dcc'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.196152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b447eb1e67432186cd0bc3f9963dcc'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.214103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 907/1818] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:47 2022] DELETE /identity/v3/users/19b447eb1e67432186cd0bc3f9963dcc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:48.219112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:48.219112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:48.219310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:48.242902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:48.244111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:48.253635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:48.354207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 912/1819] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:48.371151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:48.373015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:48.395866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:48.440371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef375514-2cb0-4a98-a0ac-54bef146f06e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:48.444023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:48.444651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:48.444967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:48.445305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:48.457215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:48.458749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6499e35-f666-4ded-9ce0-a174692e2871 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:48.468853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 908/1820] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:48.801309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:48.803036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:48.826107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768af60f-1930-41ba-a2bd-536a011d42a0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:48.834197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:48.835033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:48.835303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:48.835581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] PATH_INFO: `/v3/projects/18b8a5cdbf314feab101e309f3b58478` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:48.837011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b8a5cdbf314feab101e309f3b58478)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:48.839188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:48.938155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b8a5cdbf314feab101e309f3b58478', 'request_id': 'req-19185fab-4956-4af0-a79c-4f218f2195cf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.944154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b8a5cdbf314feab101e309f3b58478', 'request_id': 'req-19185fab-4956-4af0-a79c-4f218f2195cf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.952433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b8a5cdbf314feab101e309f3b58478', 'request_id': 'req-19185fab-4956-4af0-a79c-4f218f2195cf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.965367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] The token cache is being invalidate because project 18b8a5cdbf314feab101e309f3b58478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:48.965969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.966785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19185fab-4956-4af0-a79c-4f218f2195cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:48.969054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 913/1821] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:48 2022] DELETE /identity/v3/projects/18b8a5cdbf314feab101e309f3b58478 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:48.973848 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:48.974136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:48.974410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.006165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:49.007741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:49.075890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 909/1822] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:49.085393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.085691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.086037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.119948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:49.121342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:49.143577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 914/1823] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:49.148125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.163714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344845f-6f45-4c68-a4dc-833f1c9dbd70 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.183765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.184708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.185107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.185395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.186959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:49.187490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.191837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.205865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 910/1824] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:49.210271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.222011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30893dab-ff3e-4e93-841f-009d9562acf7 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.232961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.233965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.234088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.234372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.235054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.250501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.298701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 915/1825] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:49.300234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.324543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330255e-e26e-478a-8951-888aec711c28 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.458274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.458990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.459359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.459706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.460518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.462819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.493922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 911/1826] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:49 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:49.498108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.510401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163d1a0-bb46-4fef-b5ab-eedff002f80a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.519375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.520207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.520631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.520973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.521898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.523551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.548178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 916/1827] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:49.551950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.564482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf824b-1027-48d8-93ba-526de096967e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.569675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.570359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.570642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.570912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.571618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.573212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e905329-1ca2-41e8-9980-7227484b5b30 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.586477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 912/1828] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:49.592357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.616692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095cdf3-9cb7-4307-8f36-344ed5053339 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.622109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.623109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.623492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.623967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] PATH_INFO: `/v3/projects/f07aa9fa036b44349598e45ec1c9954a/users/d3cadee78d014b6fb6ad1cf1503fbea1/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.645020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07aa9fa036b44349598e45ec1c9954a, user_id=d3cadee78d014b6fb6ad1cf1503fbea1, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.648122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:49.663693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 917/1829] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:49 2022] PUT /identity/v3/projects/f07aa9fa036b44349598e45ec1c9954a/users/d3cadee78d014b6fb6ad1cf1503fbea1/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:49.676480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.677026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.677255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.716964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] MFA Rules not processed for user `d3cadee78d014b6fb6ad1cf1503fbea1`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:49.718426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:49.744241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:49.820568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 913/1830] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:49.825355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:49.885962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:49.952922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ce64b-d0bb-4786-9a47-ded16d30f384 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:49.957807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:49.958463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:49.958743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:49.959022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:49.959718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:49.963222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.019906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 918/1831] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:50.022535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.044816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b100-b5c2-44da-9966-f4f7d0281fbc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.178861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.179520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.179918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.180195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.180876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.185014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.234438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 914/1832] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:50 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:50.239237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.255423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e79ab-0c04-4575-be7e-3418ebbe3b88 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.262892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.263605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.263888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.264207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.265453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.267124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.293652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 919/1833] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:50.297640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.313165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a350051-eb70-4160-807c-2c45d798697b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.322388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.323084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.323371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.323652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] PATH_INFO: `/v3/projects/16441d0be6604936b456a3fb95b297db/users/c98d7714b94347778dba8f40930a1052/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.377317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16441d0be6604936b456a3fb95b297db, user_id=c98d7714b94347778dba8f40930a1052, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.380057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.433161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 915/1834] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:50 2022] PUT /identity/v3/projects/16441d0be6604936b456a3fb95b297db/users/c98d7714b94347778dba8f40930a1052/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:50.440360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.459813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70e29-a117-4387-b780-b41e31746a24 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.609621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.610631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.611144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.611710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.612842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.615418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f87cf-b815-4282-801d-a2e0f6c3bf38 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.630002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 920/1835] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:50.634516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.646666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec93c10-899b-4c8b-ad45-306df69d5955 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.651599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.652335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.652686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.653079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] PATH_INFO: `/v3/projects/16441d0be6604936b456a3fb95b297db/users/c98d7714b94347778dba8f40930a1052/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.655776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16441d0be6604936b456a3fb95b297db, user_id=c98d7714b94347778dba8f40930a1052, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.658638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:50.675921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 916/1836] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:50 2022] PUT /identity/v3/projects/16441d0be6604936b456a3fb95b297db/users/c98d7714b94347778dba8f40930a1052/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:50.686918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.687209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.687526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.724427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] MFA Rules not processed for user `c98d7714b94347778dba8f40930a1052`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:50.725723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:50.802433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 921/1837] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:50.815555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:50.817561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:50.909122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9776c2-8eeb-4215-a730-c4b64082f188 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:50.915841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:50.916773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:50.917063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:50.917350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:50.947936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:50.992984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:50.994723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:51.007259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 917/1838] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:52.991576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:52.993069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:53.053643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:53.212763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a591f-fdb5-4062-bdb8-98a4d39b4c75 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:53.236643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:53.238397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:53.238397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:53.238625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] PATH_INFO: `/v3/users/af6703e3a63d4846848d275e464b94b8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:53.241008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6703e3a63d4846848d275e464b94b8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:53.243135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:53.393567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6703e3a63d4846848d275e464b94b8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.394348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6703e3a63d4846848d275e464b94b8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.419561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6703e3a63d4846848d275e464b94b8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.454549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:53.456115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:53.464390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6703e3a63d4846848d275e464b94b8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.474203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-efc06b58-9fb9-40e4-92ab-e4fb7ee0b98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6703e3a63d4846848d275e464b94b8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.492971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 922/1839] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:52 2022] DELETE /identity/v3/users/af6703e3a63d4846848d275e464b94b8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:53.495131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a146-b66c-4065-9c6c-89177e681b61 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:53.501423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:53.501950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:53.502256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:53.533714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:53.535092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:53.673143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:53.674974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:53.675751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:53.676466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] PATH_INFO: `/v3/users/3cc3d87c8d8144e89699dd99ef61bb96` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:53.678729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc3d87c8d8144e89699dd99ef61bb96)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:53.681944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:53.693360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 923/1840] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:53.716715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:53.719366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:53.770476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc3d87c8d8144e89699dd99ef61bb96'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.782721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:53.788326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc3d87c8d8144e89699dd99ef61bb96'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.789712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc3d87c8d8144e89699dd99ef61bb96'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.800433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bacad-1ee1-4376-bcac-906682f860fc None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:53.804900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:53.805546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:53.805975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:53.806257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:53.808562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:53.819884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc3d87c8d8144e89699dd99ef61bb96'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.820635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc3d87c8d8144e89699dd99ef61bb96'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:53.828418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:53.828966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f029a33-4dc5-4f50-91be-3bd264696657 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:53.843766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 924/1841] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:53.847462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 918/1842] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:53 2022] DELETE /identity/v3/users/3cc3d87c8d8144e89699dd99ef61bb96 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:53.852515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:53.852864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:53.853157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:53.882122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:53.883545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.016639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:54.018366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.034012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 925/1843] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:54.038551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aab9f-6156-47c6-91b5-97badf229548 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:54.046898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:54.047851 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:54.049021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:54.049021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] PATH_INFO: `/v3/users/d3cadee78d014b6fb6ad1cf1503fbea1` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:54.050231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3cadee78d014b6fb6ad1cf1503fbea1)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:54.053775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:54.088921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:54.091871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.146873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:54.180597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3cadee78d014b6fb6ad1cf1503fbea1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:54.196012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3cadee78d014b6fb6ad1cf1503fbea1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:54.196772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3cadee78d014b6fb6ad1cf1503fbea1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:54.214770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bff2-f9a4-4f7a-9cb0-8bef149c0f87 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:54.217177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3cadee78d014b6fb6ad1cf1503fbea1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:54.219840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-66e21219-7709-4007-a3bf-82048a5e0729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3cadee78d014b6fb6ad1cf1503fbea1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:54.224722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:54.225645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:54.226494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:54.228875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:54.231552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.253864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 919/1844] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:54 2022] DELETE /identity/v3/users/d3cadee78d014b6fb6ad1cf1503fbea1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:54.258393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:54.272076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:54.272076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:54.277404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:54.279519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:54.294764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 926/1845] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:54.295280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:54.298111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.378243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 920/1846] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:54.415392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:54.417065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:54.429093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bf5-22e6-4619-8cad-de5b5ce24148 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:54.511365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:54.512585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:54.513450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:54.513985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:54.535484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:54.538560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:54.557012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 927/1847] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:55.571270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:55.573175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:55.588197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9461fb-88a5-4663-bf87-fa7c7c60d796 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:55.608517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:55.609279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:55.609457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:55.609728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] PATH_INFO: `/v3/projects/ad47d1e9abac472bb18ce63879e9def2` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:55.611436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad47d1e9abac472bb18ce63879e9def2)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:55.613360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:55.778930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad47d1e9abac472bb18ce63879e9def2', 'request_id': 'req-aa06aff8-7061-4126-9520-7b459df38a68'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.787706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad47d1e9abac472bb18ce63879e9def2', 'request_id': 'req-aa06aff8-7061-4126-9520-7b459df38a68'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.817210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad47d1e9abac472bb18ce63879e9def2', 'request_id': 'req-aa06aff8-7061-4126-9520-7b459df38a68'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.831889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] The token cache is being invalidate because project ad47d1e9abac472bb18ce63879e9def2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:55.832310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.833107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.838075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 921/1848] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:55 2022] DELETE /identity/v3/projects/ad47d1e9abac472bb18ce63879e9def2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:55.840990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:55.852412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40e50a-216d-4db1-afed-9e783c3f9d67 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:55.895631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:55.930167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:55.930938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:55.931220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:55.931499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] PATH_INFO: `/v3/users/abc6bcd2c747408296de4e63c669e1ef` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:55.932667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc6bcd2c747408296de4e63c669e1ef)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:55.934623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:55.994118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc6bcd2c747408296de4e63c669e1ef'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:55.994900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc6bcd2c747408296de4e63c669e1ef'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.025318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc6bcd2c747408296de4e63c669e1ef'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.041071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc6bcd2c747408296de4e63c669e1ef'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.048917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc6bcd2c747408296de4e63c669e1ef'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.055938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 928/1849] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:55 2022] DELETE /identity/v3/users/abc6bcd2c747408296de4e63c669e1ef => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:56.241124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:56.242001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:56.256004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06aff8-7061-4126-9520-7b459df38a68 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:56.303754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:56.359978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:56.362334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:56.362807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:56.364000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] PATH_INFO: `/v3/projects/5e0673cad8e1418a92407e2b9fa55a8c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:56.366716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0673cad8e1418a92407e2b9fa55a8c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:56.370794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:56.440229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:56.441802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:56.468456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11409a05-a826-4555-9af7-c5af6190027e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:56.484249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:56.485110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:56.485572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:56.485984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] PATH_INFO: `/v3/projects/925ea98a59144d38b4d649895056a76d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:56.487508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=925ea98a59144d38b4d649895056a76d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:56.489569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:56.510776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0673cad8e1418a92407e2b9fa55a8c', 'request_id': 'req-9e0c4b17-c234-4887-9da4-664c9b06c8b5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.519652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0673cad8e1418a92407e2b9fa55a8c', 'request_id': 'req-9e0c4b17-c234-4887-9da4-664c9b06c8b5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.530265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0673cad8e1418a92407e2b9fa55a8c', 'request_id': 'req-9e0c4b17-c234-4887-9da4-664c9b06c8b5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.598659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] The token cache is being invalidate because project 5e0673cad8e1418a92407e2b9fa55a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:56.599292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.600092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925ea98a59144d38b4d649895056a76d', 'request_id': 'req-eba94cc7-62ae-4740-8ad8-f5e07877d06e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.601079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.609362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 922/1850] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:56 2022] DELETE /identity/v3/projects/5e0673cad8e1418a92407e2b9fa55a8c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:56.613437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:56.616055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925ea98a59144d38b4d649895056a76d', 'request_id': 'req-eba94cc7-62ae-4740-8ad8-f5e07877d06e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.621927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:56.652508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925ea98a59144d38b4d649895056a76d', 'request_id': 'req-eba94cc7-62ae-4740-8ad8-f5e07877d06e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.678416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] The token cache is being invalidate because project 925ea98a59144d38b4d649895056a76d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:56.678774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.679622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.684936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 929/1851] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:56 2022] DELETE /identity/v3/projects/925ea98a59144d38b4d649895056a76d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:56.686634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:56.690109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:56.753126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4b17-c234-4887-9da4-664c9b06c8b5 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:56.763449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:56.826166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94cc7-62ae-4740-8ad8-f5e07877d06e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:56.833393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:56.834296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:56.835255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:56.835448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:56.835827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] PATH_INFO: `/v3/users/b0b8a00be5554bd4aa83f43d310a5df9` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:56.836047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:56.836339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:56.836620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] PATH_INFO: `/v3/projects/f07aa9fa036b44349598e45ec1c9954a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:56.837735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b8a00be5554bd4aa83f43d310a5df9)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:56.837899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07aa9fa036b44349598e45ec1c9954a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:56.839822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:56.840208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:56.875406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:56.930940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07aa9fa036b44349598e45ec1c9954a', 'request_id': 'req-8abc4e58-fa09-43f2-a7f5-674aabb26649'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.942342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07aa9fa036b44349598e45ec1c9954a', 'request_id': 'req-8abc4e58-fa09-43f2-a7f5-674aabb26649'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.944904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b8a00be5554bd4aa83f43d310a5df9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.946385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b8a00be5554bd4aa83f43d310a5df9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.959083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b8a00be5554bd4aa83f43d310a5df9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.963385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07aa9fa036b44349598e45ec1c9954a', 'request_id': 'req-8abc4e58-fa09-43f2-a7f5-674aabb26649'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.976816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] The token cache is being invalidate because project f07aa9fa036b44349598e45ec1c9954a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:56.977148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.977883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-8abc4e58-fa09-43f2-a7f5-674aabb26649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:56.980602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 923/1852] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:56 2022] DELETE /identity/v3/projects/f07aa9fa036b44349598e45ec1c9954a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:56.982293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4da8-46da-469e-82db-23cd457efa8d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:56.982293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4da8-46da-469e-82db-23cd457efa8d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:56.982513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4da8-46da-469e-82db-23cd457efa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:56.994512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b8a00be5554bd4aa83f43d310a5df9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.003561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b8a00be5554bd4aa83f43d310a5df9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.011370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 930/1853] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:56 2022] DELETE /identity/v3/users/b0b8a00be5554bd4aa83f43d310a5df9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:57.012146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.012828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-8d3a4da8-46da-469e-82db-23cd457efa8d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:57.013728 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.014894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8d3a4da8-46da-469e-82db-23cd457efa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.136570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 924/1854] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:57.146670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.146957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.147237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.164020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb68a1-cd98-4672-aaa6-b9e7b873a97a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.170889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.171715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.172021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.172303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] PATH_INFO: `/v3/users/c98d7714b94347778dba8f40930a1052` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.176525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c98d7714b94347778dba8f40930a1052)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.178545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:57.178661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.180197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.190422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 925/1855] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:57.194227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.206887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d195b-bc85-4cd5-97c8-de3b03405a8e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.213176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.213934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.214287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.214632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.215554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:57.216808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.218668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.237868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 926/1856] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:50:57.246985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.271591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c98d7714b94347778dba8f40930a1052'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.273444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c98d7714b94347778dba8f40930a1052'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.275314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53efb8-2c77-4c7f-9223-904d499091cf admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.281157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.282966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.283440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.283946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.284871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.288086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65a12a-b4d1-4c5a-a6c8-2211d58bc8b0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.310212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c98d7714b94347778dba8f40930a1052'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.314343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:57.335239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 927/1857] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:57 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:57.336043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c98d7714b94347778dba8f40930a1052'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.336514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17386-1c7d-476d-8984-7126fc5826e0 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.336810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17386-1c7d-476d-8984-7126fc5826e0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.337089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a17386-1c7d-476d-8984-7126fc5826e0 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.339087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 928/1858] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:50:57.339652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a17386-1c7d-476d-8984-7126fc5826e0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.349430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fb87e1e8-ab37-47f2-807e-34d6501b3683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c98d7714b94347778dba8f40930a1052'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:57.370799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 931/1859] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:50:57 2022] DELETE /identity/v3/users/c98d7714b94347778dba8f40930a1052 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:57.372920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.373324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.374382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.405622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:57.406830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.490660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a17386-1c7d-476d-8984-7126fc5826e0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.505834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:57.575817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 932/1860] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:50:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:57.582681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.583410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.583777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.584145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.585400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.587502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.616998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.618713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.639710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65fbdd-42be-41ac-886e-3d7eb5271f2c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.645872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.646515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.647607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 929/1861] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:57 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:57.650732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.651033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.653010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.658465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.677645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7db7-a65f-467e-b74b-2c00e1310a36 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.678419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.680289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.690064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.690064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.690229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.690369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.691113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.693063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 933/1862] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:50:57.693259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.718171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 930/1863] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:57.722543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.740301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8e56b-3757-4300-8183-7848249432e3 service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.753681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.754368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.754647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.754922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.755625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.757236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c485-807f-4d25-8f13-2e47751db4c9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.769145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 934/1864] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:57.773307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.786104 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e3b8-9e51-49f2-857a-31f54f2f419b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.792179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.793009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.793390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.793765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] PATH_INFO: `/v3/projects/7ea677692fc244709a5a75a8f603b64d/users/9b3bbcda7036490188306f7cc0b45ea0/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.818258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea677692fc244709a5a75a8f603b64d, user_id=9b3bbcda7036490188306f7cc0b45ea0, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.821037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.838111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 931/1865] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:57 2022] PUT /identity/v3/projects/7ea677692fc244709a5a75a8f603b64d/users/9b3bbcda7036490188306f7cc0b45ea0/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:57.848966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.849249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.849813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.885573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] MFA Rules not processed for user `9b3bbcda7036490188306f7cc0b45ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:57.887049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.889613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.890279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:57.968982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce746-cd68-44a6-acf6-8384803f573e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:57.975030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:57.975698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:57.976018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:57.976303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] PATH_INFO: `/v3/projects/16441d0be6604936b456a3fb95b297db` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:57.977682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16441d0be6604936b456a3fb95b297db)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:57.979630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:57.988373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 935/1866] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:57.989043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:57.990347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.004425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1333c-6ec6-4248-9799-ebebef2c9a7e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.009756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.010568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.010876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.011154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] PATH_INFO: `/v3/projects/3612f0db272941f7a6b7ea0c950a6097` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.012343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3612f0db272941f7a6b7ea0c950a6097)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.015006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.032181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16441d0be6604936b456a3fb95b297db', 'request_id': 'req-5ae994af-a652-4d3d-94c6-f81778b4571d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.038283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16441d0be6604936b456a3fb95b297db', 'request_id': 'req-5ae994af-a652-4d3d-94c6-f81778b4571d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.046262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16441d0be6604936b456a3fb95b297db', 'request_id': 'req-5ae994af-a652-4d3d-94c6-f81778b4571d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.047537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.053838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.059591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] The token cache is being invalidate because project 16441d0be6604936b456a3fb95b297db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:58.059875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.060805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.063427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 932/1867] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:57 2022] DELETE /identity/v3/projects/16441d0be6604936b456a3fb95b297db => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:58.064257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.068028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.068196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3612f0db272941f7a6b7ea0c950a6097', 'request_id': 'req-b09508bf-d037-4910-ad9a-9fc69e08c8a7'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.073907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3612f0db272941f7a6b7ea0c950a6097', 'request_id': 'req-b09508bf-d037-4910-ad9a-9fc69e08c8a7'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.083456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3612f0db272941f7a6b7ea0c950a6097', 'request_id': 'req-b09508bf-d037-4910-ad9a-9fc69e08c8a7'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.098053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] The token cache is being invalidate because project 3612f0db272941f7a6b7ea0c950a6097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:50:58.098466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.099285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b09508bf-d037-4910-ad9a-9fc69e08c8a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:50:58.101272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 936/1868] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:50:57 2022] DELETE /identity/v3/projects/3612f0db272941f7a6b7ea0c950a6097 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:58.102410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86485-5bea-4ac4-99ae-e51ade7c5e82 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.102762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86485-5bea-4ac4-99ae-e51ade7c5e82 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.103119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86485-5bea-4ac4-99ae-e51ade7c5e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.132996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4fe86485-5bea-4ac4-99ae-e51ade7c5e82 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:58.134234 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4fe86485-5bea-4ac4-99ae-e51ade7c5e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.141869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae994af-a652-4d3d-94c6-f81778b4571d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.210877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 937/1869] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:58.211858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edff7d-5e67-4ccb-a47e-1be99864ac31 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.212352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edff7d-5e67-4ccb-a47e-1be99864ac31 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.212705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edff7d-5e67-4ccb-a47e-1be99864ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.231914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.234375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.234634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.235352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.236080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.239873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27f428-8fd6-4e41-9cd6-638f01771ede admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.253781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-52edff7d-5e67-4ccb-a47e-1be99864ac31 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:58.255103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-52edff7d-5e67-4ccb-a47e-1be99864ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.278204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 938/1870] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:58.279484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.280386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.280668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.302949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 933/1871] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.305455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.305746 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.306055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.324867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:58.330389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.339868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:58.341022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.414648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 939/1872] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:58.418711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.419987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.422338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 934/1873] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:58.423128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.432680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acc439-fe1f-476f-bde1-95fbe25cdd65 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.435206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1847e-0f64-418b-a0ba-f605b9324e77 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.437625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.438344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.438713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.439061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.439746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.439870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:58.440246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.440362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.440638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.440919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.441582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.442062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.443572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.454408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 935/1874] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:58.455214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.465732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd3566-fcce-441f-9b0d-ed34816df281 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.469876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 940/1875] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.469876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.473484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.474129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.474407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.474673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.475364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:50:58.475667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.477336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.478157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23fa0-75f7-4940-8f50-f531fab827c5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.482948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.483543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.483828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.484107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.484821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.486738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.493435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 936/1876] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:50:58.493435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.504930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.515352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98a79c-bdd6-4e6f-b14a-2df5c60f9c36 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.522389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.523041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.523416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.523703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.524926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.525031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 941/1877] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.526441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.530495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.539613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.544571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 937/1878] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.545968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.556161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a623-0356-4efd-b9bc-d9d9da564a41 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.603347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f310-0486-4e05-8fe6-0afa86724291 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.608793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.609424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.609710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.610151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.610826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.612793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.621329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.622069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.622420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.622769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.623569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.625500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.643289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 942/1879] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.644425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.653201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 938/1880] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.654182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.665147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea1780-34e2-4ad1-b16e-f2126a170e25 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.728732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.729487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.729887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.730237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.730987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.733609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.740262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76efa9-6991-4813-885b-7a0699231ffc admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.746507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.747324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.747649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.747962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.748835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.750652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.757936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 939/1881] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:58.758931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.762815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 943/1882] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.763866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.770270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395c84f-2ca1-4bf4-bbfa-a42bcff29439 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.776167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518443ba-5580-497d-b345-ce3f953befde admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.776467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.777185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.777468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.777850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.778568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.780232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.783282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.783922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.784197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.784475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.785178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.787263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.790993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 940/1883] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.791798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.802816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d11b0-50c4-4e07-b769-c895b53ddbae admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.804945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 944/1884] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.806457 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.809304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.810014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.810299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.810661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] PATH_INFO: `/v3/projects/8c784b3e3609463e88ba2f73ac28cc9e/users/cb27ed12630145558553ebbd41e31647/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.818766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613510cc-a400-4f4d-aef1-cb55c7925b81 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.824590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.825232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.825506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.825780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.826538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.828261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.828631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c784b3e3609463e88ba2f73ac28cc9e, user_id=cb27ed12630145558553ebbd41e31647, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.831372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.842827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 945/1885] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.842827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.850599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f75844-1932-4ea2-ad51-527cc4a66e95 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.855036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 941/1886] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:58 2022] PUT /identity/v3/projects/8c784b3e3609463e88ba2f73ac28cc9e/users/cb27ed12630145558553ebbd41e31647/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:58.858098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:58.902289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.910846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.911479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.911765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.912048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.912749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.915207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3984-5180-407d-afd6-3ff263a9fafc admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.925520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b141d7-0431-490c-98a1-4be7f6f4e8f9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:58.927482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 946/1887] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:58.929268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b340649-f7d3-4c48-b37c-a3dfa554ad83 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.929915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b340649-f7d3-4c48-b37c-a3dfa554ad83 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.930197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:58.930296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b340649-f7d3-4c48-b37c-a3dfa554ad83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.930875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:58.931190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:58.931469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] PATH_INFO: `/v3/projects/13c13c269e5f46e1a8236c89377c2171/users/a1c132b2fdc14e7ca4ffb18109a0fa0d/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:58.939624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:58.947860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c13c269e5f46e1a8236c89377c2171, user_id=a1c132b2fdc14e7ca4ffb18109a0fa0d, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:58.950721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:58.961308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2b340649-f7d3-4c48-b37c-a3dfa554ad83 None None] MFA Rules not processed for user `cb27ed12630145558553ebbd41e31647`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:58.962575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2b340649-f7d3-4c48-b37c-a3dfa554ad83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:58.964856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 942/1888] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:58 2022] PUT /identity/v3/projects/13c13c269e5f46e1a8236c89377c2171/users/a1c132b2fdc14e7ca4ffb18109a0fa0d/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:58.965751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.036270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd2498-1385-4f49-af62-ea3be7ed03c0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.038263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 947/1889] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:59.039948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671175a-6bac-4006-b473-47e879a10263 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.040271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671175a-6bac-4006-b473-47e879a10263 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.040553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671175a-6bac-4006-b473-47e879a10263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.044285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.044987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.045350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.045635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] PATH_INFO: `/v3/projects/b7700389fc514a1f806c68121f49e89e/users/47d77e80b50f418e8b1386a68c3f3106/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.059558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b671175a-6bac-4006-b473-47e879a10263 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:59.061936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b671175a-6bac-4006-b473-47e879a10263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:59.063638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7700389fc514a1f806c68121f49e89e, user_id=47d77e80b50f418e8b1386a68c3f3106, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.067726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.072608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 948/1890] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:59.076632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671175a-6bac-4006-b473-47e879a10263 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.085154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671175a-6bac-4006-b473-47e879a10263 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.087555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 943/1891] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:58 2022] PUT /identity/v3/projects/b7700389fc514a1f806c68121f49e89e/users/47d77e80b50f418e8b1386a68c3f3106/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:59.088419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.090185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:59.153992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.154681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.154966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.155244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.155918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.156893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05eb8-6f79-4ba7-8100-369ee7a3152d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.157857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.160669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.161361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.161715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.162081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.173712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.175535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec366cc-bdda-48f1-b16b-d74fc7d949a2 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.187288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 944/1892] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:50:59.188587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.188993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.189151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.191848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 949/1893] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:59.195574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.225753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] MFA Rules not processed for user `47d77e80b50f418e8b1386a68c3f3106`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:59.227123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:59.269733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:59.271607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff61f0-1811-43b3-9deb-2de2edc32a37 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.277478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.278337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.278954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.279301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.280122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.282355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.309767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 950/1894] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:59.311394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.315515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 945/1895] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:50:59.317374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:59.321038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.323587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe60b-e1c8-49e9-a9a1-76e849ce0b37 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.331519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.332560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.332991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.333270 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862636ab-9e8e-4581-82d7-f3b321713c7a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.333873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.334754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.336590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.339213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.339920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.340281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.340725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.341566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.343962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.352379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 951/1896] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:59.356585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.371404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42218fc-7c56-4594-ab41-ed45e2030759 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.377067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.377733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.378047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.378370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] PATH_INFO: `/v3/projects/3581bdd9d9ae41da88d06a39da04c413/users/edfe029861d0472090711927587d5a50/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.389875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 946/1897] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:50:59.393443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.396397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3581bdd9d9ae41da88d06a39da04c413, user_id=edfe029861d0472090711927587d5a50, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.399257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.423050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 952/1898] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:59 2022] PUT /identity/v3/projects/3581bdd9d9ae41da88d06a39da04c413/users/edfe029861d0472090711927587d5a50/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:59.425447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.478786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe474ca-fc3a-4ae8-acbb-2f4fc10ccf55 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.503535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc686ec-987e-414e-89bb-67aa68ba4a5a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.510116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.511682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.511980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.512259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.512977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.515974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.546934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 953/1899] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:50:59.549275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.563140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072e982-1e54-4837-8c63-daaebe2375c2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.569420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.570166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.570777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.571125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] PATH_INFO: `/v3/projects/3581bdd9d9ae41da88d06a39da04c413/users/edfe029861d0472090711927587d5a50/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.575524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3581bdd9d9ae41da88d06a39da04c413, user_id=edfe029861d0472090711927587d5a50, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.579579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ff2c6-c172-429e-afe4-4bd18b0b34b3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.594750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.594984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 954/1900] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:59 2022] PUT /identity/v3/projects/3581bdd9d9ae41da88d06a39da04c413/users/edfe029861d0472090711927587d5a50/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:50:59.595705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.595985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.596313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.596978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.599115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.606119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.606777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.607268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.623983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 947/1901] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:50:59.628208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.646307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7230-3878-4aae-a21b-eca201850747 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.678157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] MFA Rules not processed for user `edfe029861d0472090711927587d5a50`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:50:59.679596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:59.715862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:59.739016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.740220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.740577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.740932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.741749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.743469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.785869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 948/1902] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:50:59.787356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.794413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 955/1903] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:50:59.806314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.808118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:50:59.808675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4d57-f675-4305-9fe5-b2fa1291f719 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.814805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.815634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.816055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.816331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] PATH_INFO: `/v3/projects/9a68f89da77c4498bdd2bcd716132914/users/29bc1ede43e34d03a49bccc65fcdab0f/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:50:59.835174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a68f89da77c4498bdd2bcd716132914, user_id=29bc1ede43e34d03a49bccc65fcdab0f, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:50:59.838458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:50:59.858885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 949/1904] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:50:59 2022] PUT /identity/v3/projects/9a68f89da77c4498bdd2bcd716132914/users/29bc1ede43e34d03a49bccc65fcdab0f/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:50:59.863316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:50:59.867388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:50:59.890910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bd1e5-0a49-4810-a868-439ce1400b4a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:50:59.967977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:50:59.968827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:50:59.969371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:50:59.969693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.030242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e986e-a92b-4f2a-be46-7aa183f4ddc0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:00.038315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:00.038954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.039246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.039503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.040214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.041833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.068766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 950/1905] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:00.073434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:00.092253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9e109-f40a-4309-9e87-0f715f58a3c9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:00.111920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.113680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e22af-4a79-41af-adff-80a230b70971 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.129738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:00.129738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.129738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.129738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] PATH_INFO: `/v3/projects/9a68f89da77c4498bdd2bcd716132914/users/29bc1ede43e34d03a49bccc65fcdab0f/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.132506 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a68f89da77c4498bdd2bcd716132914, user_id=29bc1ede43e34d03a49bccc65fcdab0f, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.135297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.140475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 956/1906] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:50:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:00.170923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 951/1907] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:00 2022] PUT /identity/v3/projects/9a68f89da77c4498bdd2bcd716132914/users/29bc1ede43e34d03a49bccc65fcdab0f/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:00.173288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.173798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.174253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.214624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:00.255414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] MFA Rules not processed for user `29bc1ede43e34d03a49bccc65fcdab0f`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:00.259704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:00.335241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efe3f8-c686-4bfd-85a9-bafbe4546901 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:00.349807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:00.350330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.350616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.350891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.357231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:00.477932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 957/1908] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:00.483206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.486339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.498256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:00.509649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 952/1909] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:00.546285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:00.567202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b2738-4537-42ff-b00e-7d10a3924f9d None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:00.575782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:00.576290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.576612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.576877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.599879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.601681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.628215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 958/1910] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:00.804734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:00.806870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:00.823966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738738a1-095f-469e-b4d3-3d95a8a6219f service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:00.828822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:00.831181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:00.831821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:00.832252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:00.958042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:00.959763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66e69b-14fa-49a4-90a0-7d59564ade2d service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:00.972364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 953/1911] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:03.383756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:03.385229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:03.402646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:03.472104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49a2d2-169c-456a-8159-4efc44072ef0 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:03.490668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:03.492613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:03.492613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:03.494020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] PATH_INFO: `/v3/users/6def97bed1a14e849f53a1a07906e111` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:03.496331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6def97bed1a14e849f53a1a07906e111)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:03.503108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:03.564310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6def97bed1a14e849f53a1a07906e111'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:03.565387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6def97bed1a14e849f53a1a07906e111'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:03.576761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6def97bed1a14e849f53a1a07906e111'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:03.595836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6def97bed1a14e849f53a1a07906e111'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:03.604933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6def97bed1a14e849f53a1a07906e111'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:03.612330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 959/1912] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:03 2022] DELETE /identity/v3/users/6def97bed1a14e849f53a1a07906e111 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:03.616716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:03.617160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:03.617261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:03.638750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:03.640608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:03.712892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 954/1913] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:03.724263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:03.726172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:03.804171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea3085-1387-454d-ac52-b12c8ad1757c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:03.809584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:03.810372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:03.811430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:03.811430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:03.824019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:03.825636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:03.837678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 960/1914] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:04.169198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:04.171439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.192494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8f59c-1a9b-4dca-8939-6ce9c2f7ecfe None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:04.207193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:04.208023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.211111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.211484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] PATH_INFO: `/v3/users/9b3bbcda7036490188306f7cc0b45ea0` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.213237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3bbcda7036490188306f7cc0b45ea0)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:04.215257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:04.312400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3bbcda7036490188306f7cc0b45ea0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.325297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3bbcda7036490188306f7cc0b45ea0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.326086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3bbcda7036490188306f7cc0b45ea0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.358583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3bbcda7036490188306f7cc0b45ea0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.359341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5f2182cb-9419-41f0-9a76-e9ddc0a38338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3bbcda7036490188306f7cc0b45ea0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.361624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:04.363242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.377675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 955/1915] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:04 2022] DELETE /identity/v3/users/9b3bbcda7036490188306f7cc0b45ea0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:04.382168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.382583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.382955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.385647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c19ce-8f79-462c-9342-7a2870f4b86f service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:04.406178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:04.408037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.414954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:04.458926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:04.509868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 956/1916] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:04.518689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:04.519370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.519844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.520179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] PATH_INFO: `/v3/users/47d77e80b50f418e8b1386a68c3f3106` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.522933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d77e80b50f418e8b1386a68c3f3106)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:04.527305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:04.555354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:04.558937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.636816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d77e80b50f418e8b1386a68c3f3106'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.636816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d77e80b50f418e8b1386a68c3f3106'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.653999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d77e80b50f418e8b1386a68c3f3106'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.664970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06f4ec-b3a7-4b77-8163-a7049d430ed3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:04.674789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:04.683762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.684948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d77e80b50f418e8b1386a68c3f3106'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.685318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.686498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.703693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:04.704790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-45374591-181b-4a3b-9fc5-f251dd536972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d77e80b50f418e8b1386a68c3f3106'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:04.707146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da8904-36a5-4fdf-8057-22ea7791e5ff service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:04.723276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 961/1917] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:04 2022] DELETE /identity/v3/users/47d77e80b50f418e8b1386a68c3f3106 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:04.730391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.730683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.730965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.733594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 962/1918] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:51:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:51:04.735431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 957/1919] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:04.735555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:04.737345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009048-9381-446c-ae6d-8fc670308fd8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.737883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009048-9381-446c-ae6d-8fc670308fd8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.738376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009048-9381-446c-ae6d-8fc670308fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.742907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.783085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-33009048-9381-446c-ae6d-8fc670308fd8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:04.785678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-33009048-9381-446c-ae6d-8fc670308fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:04.870508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e06c04-e635-4f2a-822a-5a75e299e57e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:04.879557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:04.884259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.884828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.885261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] PATH_INFO: `/v3/projects/f4463aa83c704417b7662b80cda75ba8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.886797 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4463aa83c704417b7662b80cda75ba8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:04.890378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:04.926029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 958/1920] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:04.929048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:04.929416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:04.929911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:04.964128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:04.968953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.009897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4463aa83c704417b7662b80cda75ba8', 'request_id': 'req-8d254eba-ac09-42f2-834c-c21f56018511'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.024984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4463aa83c704417b7662b80cda75ba8', 'request_id': 'req-8d254eba-ac09-42f2-834c-c21f56018511'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.035297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:05.049291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4463aa83c704417b7662b80cda75ba8', 'request_id': 'req-8d254eba-ac09-42f2-834c-c21f56018511'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.084898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] The token cache is being invalidate because project f4463aa83c704417b7662b80cda75ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:05.086130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.087187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.098449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 963/1921] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:04 2022] DELETE /identity/v3/projects/f4463aa83c704417b7662b80cda75ba8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:05.100667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:05.106431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.135514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:05.178620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 959/1922] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:05.181578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:05.184927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.221334 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdbf69-3121-4097-93c5-c67e695b10e2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:05.240738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d254eba-ac09-42f2-834c-c21f56018511 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:05.246628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:05.247652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:05.247937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:05.248843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:05.252281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.319874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:05.321281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:05.322338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:05.322807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] PATH_INFO: `/v3/users/ec7465e03c3643678d5181b79d128a4a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:05.324778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7465e03c3643678d5181b79d128a4a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:05.328074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:05.374403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:05.378133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:05.396891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 964/1923] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:05.397695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:05.413299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.449259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7465e03c3643678d5181b79d128a4a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.466053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7465e03c3643678d5181b79d128a4a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.473018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7465e03c3643678d5181b79d128a4a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.502450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7465e03c3643678d5181b79d128a4a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.503921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7465e03c3643678d5181b79d128a4a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.542602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99150db3-b51e-497f-a94f-25ae744ed1ba service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:05.551704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:05.552443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:05.552914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:05.553233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:05.557062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.574118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:05.576254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:05.578886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 960/1924] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:05 2022] DELETE /identity/v3/users/ec7465e03c3643678d5181b79d128a4a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:05.601670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 965/1925] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:05.686853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:05.692772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:05.864317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44097d9b-0432-4992-963e-70b652abd171 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:05.870735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:05.871632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:05.871991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:05.872336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] PATH_INFO: `/v3/projects/7ea677692fc244709a5a75a8f603b64d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:05.874243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea677692fc244709a5a75a8f603b64d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:05.876708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:05.889940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:05.983776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea677692fc244709a5a75a8f603b64d', 'request_id': 'req-7d0ba369-9347-48da-b39b-ae397280488e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:05.994308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea677692fc244709a5a75a8f603b64d', 'request_id': 'req-7d0ba369-9347-48da-b39b-ae397280488e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.010147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea677692fc244709a5a75a8f603b64d', 'request_id': 'req-7d0ba369-9347-48da-b39b-ae397280488e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.026754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] The token cache is being invalidate because project 7ea677692fc244709a5a75a8f603b64d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:06.027118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.028287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.032094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 961/1926] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:05 2022] DELETE /identity/v3/projects/7ea677692fc244709a5a75a8f603b64d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:06.038380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:06.065237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51299f-4339-4e78-9fd3-9e4fe1b0207c service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:06.090459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:06.124864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:06.126999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:06.236992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:06.239001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:06.239779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:06.243218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] PATH_INFO: `/v3/users/cb27ed12630145558553ebbd41e31647` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:06.246272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb27ed12630145558553ebbd41e31647)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:06.248321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:06.319845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba369-9347-48da-b39b-ae397280488e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:06.336943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:06.337933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:06.338706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:06.339113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] PATH_INFO: `/v3/projects/b7700389fc514a1f806c68121f49e89e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:06.342737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7700389fc514a1f806c68121f49e89e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:06.347593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:06.412970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb27ed12630145558553ebbd41e31647'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.414956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb27ed12630145558553ebbd41e31647'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.432449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb27ed12630145558553ebbd41e31647'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.449846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7700389fc514a1f806c68121f49e89e', 'request_id': 'req-5035ebda-6aee-4823-99a5-075d42984377'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.456481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7700389fc514a1f806c68121f49e89e', 'request_id': 'req-5035ebda-6aee-4823-99a5-075d42984377'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.458961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb27ed12630145558553ebbd41e31647'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.473082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7700389fc514a1f806c68121f49e89e', 'request_id': 'req-5035ebda-6aee-4823-99a5-075d42984377'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.473899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb27ed12630145558553ebbd41e31647'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.494865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 966/1927] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:06 2022] DELETE /identity/v3/users/cb27ed12630145558553ebbd41e31647 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:06.496901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:06.503707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:06.509921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] The token cache is being invalidate because project b7700389fc514a1f806c68121f49e89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:06.510219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.514064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.518083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 962/1928] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:06 2022] DELETE /identity/v3/projects/b7700389fc514a1f806c68121f49e89e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:06.523333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:06.529107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:06.626399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:06.718058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108078-61bf-4718-a8bf-8608943e79f7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:06.734528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:06.738226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:06.739578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:06.740096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] PATH_INFO: `/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:06.746134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e57f03d6e241a397bd4dbe3e96d5d0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:06.751729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:06.753155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035ebda-6aee-4823-99a5-075d42984377 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:06.767442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:06.770644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:06.771138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:06.772149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] PATH_INFO: `/v3/users/29bc1ede43e34d03a49bccc65fcdab0f` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:06.776862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bc1ede43e34d03a49bccc65fcdab0f)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:06.782807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:06.898779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:06.921527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e57f03d6e241a397bd4dbe3e96d5d0', 'request_id': 'req-0dfed307-9500-41d1-9a57-d8a703208a0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.930408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bc1ede43e34d03a49bccc65fcdab0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.939264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e57f03d6e241a397bd4dbe3e96d5d0', 'request_id': 'req-0dfed307-9500-41d1-9a57-d8a703208a0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.948225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bc1ede43e34d03a49bccc65fcdab0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.950704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bc1ede43e34d03a49bccc65fcdab0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:06.959245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e57f03d6e241a397bd4dbe3e96d5d0', 'request_id': 'req-0dfed307-9500-41d1-9a57-d8a703208a0a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.013586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] The token cache is being invalidate because project 98e57f03d6e241a397bd4dbe3e96d5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:07.015288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bc1ede43e34d03a49bccc65fcdab0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.022002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.022840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bc1ede43e34d03a49bccc65fcdab0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.025911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0dfed307-9500-41d1-9a57-d8a703208a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.030787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 967/1929] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:06 2022] DELETE /identity/v3/projects/98e57f03d6e241a397bd4dbe3e96d5d0 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:07.037700 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10913-245a-4f90-a21a-aa13f76a5a5e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.039978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10913-245a-4f90-a21a-aa13f76a5a5e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.040805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10913-245a-4f90-a21a-aa13f76a5a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.078345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 963/1930] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:06 2022] DELETE /identity/v3/users/29bc1ede43e34d03a49bccc65fcdab0f => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:07.097000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8fb10913-245a-4f90-a21a-aa13f76a5a5e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:07.098557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8fb10913-245a-4f90-a21a-aa13f76a5a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.184918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.187252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.221193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 968/1931] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:07.229673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.230198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.230478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.283634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:07.290038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.317249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 969/1932] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:07.319224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.363559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ba9c7-fbc5-4327-9ce4-8fd078b299ad None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.383400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.385512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.386043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.387827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.390280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:07.392150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.397845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2dd27-4b55-4c45-a20e-c133d9a4d52a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.398565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.412971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.416481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.416922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.417208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] PATH_INFO: `/v3/projects/8c784b3e3609463e88ba2f73ac28cc9e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.421149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c784b3e3609463e88ba2f73ac28cc9e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.424667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.449896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 970/1933] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:07.455487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.477211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9937472-2e46-465c-904d-60212ac7d7d2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.493549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.494291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.494578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.496096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.496792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.500739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.539939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:07.541912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c784b3e3609463e88ba2f73ac28cc9e', 'request_id': 'req-b485b41e-5299-47fe-b790-bf4150c0bd29'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.552577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c784b3e3609463e88ba2f73ac28cc9e', 'request_id': 'req-b485b41e-5299-47fe-b790-bf4150c0bd29'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.559369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 971/1934] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:07 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:07.561787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.568380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c784b3e3609463e88ba2f73ac28cc9e', 'request_id': 'req-b485b41e-5299-47fe-b790-bf4150c0bd29'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.605784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] The token cache is being invalidate because project 8c784b3e3609463e88ba2f73ac28cc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:07.606488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.610961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.613561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 964/1935] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:07 2022] DELETE /identity/v3/projects/8c784b3e3609463e88ba2f73ac28cc9e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:07.619525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.623896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.637532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3728f-467a-4bd7-a41b-d36d444baab8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.645983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.646908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.647255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.647658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.648672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.651358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.680054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 972/1936] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:07 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:07.684141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.687583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.705507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d440f-4404-4e1d-b215-1ea6916eb455 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.710774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.711515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.711909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.712285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.713211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.714967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.738901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 973/1937] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:07.740603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.742328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b41e-5299-47fe-b790-bf4150c0bd29 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.752696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.753183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271279-969c-4bf9-b750-f05dfa637e59 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.753430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.753919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.754565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] PATH_INFO: `/v3/users/edfe029861d0472090711927587d5a50` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.756056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfe029861d0472090711927587d5a50)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.758152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.758407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.759365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.759785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.760154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.760960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.763524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.765996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:07.787042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 974/1938] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:07.789927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.803290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2ef7-de1d-4ed5-acfc-6d9cbd349815 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:07.809028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:07.810039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.810518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.811681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] PATH_INFO: `/v3/projects/0ca2bad3de4e4e6f8e2a30053c3989b1/users/efd92fe270e1429b87d12f9bf093dec1/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.829067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfe029861d0472090711927587d5a50'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.831063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca2bad3de4e4e6f8e2a30053c3989b1, user_id=efd92fe270e1429b87d12f9bf093dec1, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:07.834776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bce5b-f7f3-449d-8f30-cfd3ecc3194b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:07.838760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfe029861d0472090711927587d5a50'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.840457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfe029861d0472090711927587d5a50'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.870058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfe029861d0472090711927587d5a50'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.870724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfe029861d0472090711927587d5a50'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:07.873014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 975/1939] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:07 2022] PUT /identity/v3/projects/0ca2bad3de4e4e6f8e2a30053c3989b1/users/efd92fe270e1429b87d12f9bf093dec1/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:07.881976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:07.882608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:07.882898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:07.893597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 965/1940] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:07 2022] DELETE /identity/v3/users/edfe029861d0472090711927587d5a50 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:07.928748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] MFA Rules not processed for user `efd92fe270e1429b87d12f9bf093dec1`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:07.930040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:07.974199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:07.977540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:08.098102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 976/1941] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:08.103143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:08.104756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:08.190049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:08.278350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425250a-6e20-46e3-82a3-e1b96b82c7d3 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:08.292361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:08.293133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.293503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.294215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] PATH_INFO: `/v3/projects/9a68f89da77c4498bdd2bcd716132914` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.296927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a68f89da77c4498bdd2bcd716132914)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:08.299021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:08.419652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4daeeb-2fe1-4d6b-8189-203fdf54a4ea None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:08.427459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a68f89da77c4498bdd2bcd716132914', 'request_id': 'req-a4b228bd-4849-4959-9729-6ff6491c6551'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:08.454476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a68f89da77c4498bdd2bcd716132914', 'request_id': 'req-a4b228bd-4849-4959-9729-6ff6491c6551'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:08.478715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a68f89da77c4498bdd2bcd716132914', 'request_id': 'req-a4b228bd-4849-4959-9729-6ff6491c6551'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:08.523075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] The token cache is being invalidate because project 9a68f89da77c4498bdd2bcd716132914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:08.524093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:08.527429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a4b228bd-4849-4959-9729-6ff6491c6551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:08.532691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 966/1942] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:07 2022] DELETE /identity/v3/projects/9a68f89da77c4498bdd2bcd716132914 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:08.542041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5829451-a1ab-4c68-af1f-c7b2a8b6df80 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.544057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5829451-a1ab-4c68-af1f-c7b2a8b6df80 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.544657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5829451-a1ab-4c68-af1f-c7b2a8b6df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.590805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f5829451-a1ab-4c68-af1f-c7b2a8b6df80 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:08.593008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f5829451-a1ab-4c68-af1f-c7b2a8b6df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:08.650054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:08.652611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.652611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.653109 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.655526 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:08.665607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:08.700308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:08.729001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 977/1943] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:08 2022] POST /identity/v3/projects => generated 373 bytes in 626 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:08.729273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 967/1944] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:08.737266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:08.740023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.740314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.740591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.743413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:08.793271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:08.795057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:08.863859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28fac-5280-4800-8d57-11e4250be8bc admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:08.871471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:08.872595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.873656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.874126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.875047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:08.877163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:08.920987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 978/1945] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:08 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:08.924487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:08.938857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5afc6-c774-42fa-950c-5610d0e8b476 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:08.946237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:08.947002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:08.949420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:08.949789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:08.950744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:08.952965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:08.958008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 968/1946] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:08.960241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:08.972572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 979/1947] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:08.973713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:08.982173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef005a-5cb3-49b8-823e-fa5c048b605a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:08.998528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2639aa-71a3-4eb4-8d5e-0563b91e7ed5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.001770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.002059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.002425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.002773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.003607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:09.010333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.011993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.013199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.014161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.014798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] PATH_INFO: `/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c/users/d585c0b792c04a0c92c54bc087e45563/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.017101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.037661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4f78d24aee4c0980aa0e9cf88a9b7c, user_id=d585c0b792c04a0c92c54bc087e45563, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.046505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.053391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 969/1948] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:09.058023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.089900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9b9d0-d390-4d1d-abef-97044c2d35d3 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.090769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 980/1949] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:08 2022] PUT /identity/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c/users/d585c0b792c04a0c92c54bc087e45563/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:09.095664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.125318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:09.216938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.217660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.218438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.218952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.219768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.222515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.262319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dede71-71dc-47e9-bbdf-37b408ff5a14 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.278968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.280489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.280784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.281059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.281959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.283712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.301897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 981/1950] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:09.302183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 970/1951] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:09.305739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.307026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.340516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1ab5-6de6-4c41-b4ef-aa9a60e9dac9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.430566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:09.469328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.471061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.471447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.471849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] PATH_INFO: `/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c/users/d585c0b792c04a0c92c54bc087e45563/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.476455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4f78d24aee4c0980aa0e9cf88a9b7c, user_id=d585c0b792c04a0c92c54bc087e45563, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.489864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841025f-d1a5-439c-b0ac-6c27cb44b245 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.491308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.502216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.504630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.506148 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.506464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.509021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.513085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2436e-ec5e-4981-8af7-b092d4351656 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.530508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 982/1952] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:09 2022] PUT /identity/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c/users/d585c0b792c04a0c92c54bc087e45563/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:09.532633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.538467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.565960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 971/1953] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:09 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:09.567571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.567867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.568147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.610269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] MFA Rules not processed for user `d585c0b792c04a0c92c54bc087e45563`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:09.611505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.641438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20bb1f-bef9-4b0e-b511-0794c9bed489 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.648083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.649566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.650057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.650961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] PATH_INFO: `/v3/projects/3581bdd9d9ae41da88d06a39da04c413` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.653612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3581bdd9d9ae41da88d06a39da04c413)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.657693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.697179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 972/1954] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:09.701131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.722902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c163003-9a9c-4831-b687-2dd68c122536 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.735721 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:09.755113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3581bdd9d9ae41da88d06a39da04c413', 'request_id': 'req-943107f5-b6c6-4276-a32f-115169480cd3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:09.761759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3581bdd9d9ae41da88d06a39da04c413', 'request_id': 'req-943107f5-b6c6-4276-a32f-115169480cd3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:09.771983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3581bdd9d9ae41da88d06a39da04c413', 'request_id': 'req-943107f5-b6c6-4276-a32f-115169480cd3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:09.786333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] The token cache is being invalidate because project 3581bdd9d9ae41da88d06a39da04c413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:09.786742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:09.787443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:09.789508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 983/1955] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:09 2022] DELETE /identity/v3/projects/3581bdd9d9ae41da88d06a39da04c413 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:09.790413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.794684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.861363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.862357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.862654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.862981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.863717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.865500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af261aa2-5262-4d7c-a57e-57fd25f09b86 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.883497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943107f5-b6c6-4276-a32f-115169480cd3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.886305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 973/1956] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:09.887617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.887910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.888191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.890667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.891316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.891600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.891876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.893716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.911933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:09.913494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.920601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:09.925524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 974/1957] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:09.926941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:09.933507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:09.953912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4947b1-cd21-495e-a612-652dedf53a75 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:09.961549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:09.962311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.962661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.962938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.963649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.966005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed4283-7e88-453f-b185-70d02702618c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.971552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:09.973116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:09.985125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 975/1958] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:09.985125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ef62-0581-460e-8231-ba0085c744df None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:09.985125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ef62-0581-460e-8231-ba0085c744df None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:09.985125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ef62-0581-460e-8231-ba0085c744df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:09.985553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 984/1959] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:09.986443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.009781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dfd9f-791f-4430-9b11-30ddc046a1ca service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.016621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.017893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.018175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.018462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] PATH_INFO: `/v3/projects/5120403609804c31b7c42db72bd81502/users/35ccc311ff6d4df1ab19ff2e0712bdea/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.020534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-86a2ef62-0581-460e-8231-ba0085c744df None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:10.024193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-86a2ef62-0581-460e-8231-ba0085c744df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:10.043075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5120403609804c31b7c42db72bd81502, user_id=35ccc311ff6d4df1ab19ff2e0712bdea, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.046965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.063315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 976/1960] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:10.065139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236aa3b-daec-4759-8d01-50edb1cadc09 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.065602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236aa3b-daec-4759-8d01-50edb1cadc09 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.066045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236aa3b-daec-4759-8d01-50edb1cadc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.098018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 985/1961] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:09 2022] PUT /identity/v3/projects/5120403609804c31b7c42db72bd81502/users/35ccc311ff6d4df1ab19ff2e0712bdea/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:10.099175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.133377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b236aa3b-daec-4759-8d01-50edb1cadc09 None None] MFA Rules not processed for user `a1c132b2fdc14e7ca4ffb18109a0fa0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:10.136630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b236aa3b-daec-4759-8d01-50edb1cadc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:10.230014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4d51-3f0f-43ce-a03f-c563bdc85cba admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.237487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 977/1962] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:10.237487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.237487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.237487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.244971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.246016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.246488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.246837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.247621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:10.248553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.250359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.273146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 986/1963] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:10.274332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.283758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] MFA Rules not processed for user `35ccc311ff6d4df1ab19ff2e0712bdea`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:10.285264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:10.290943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:10.320796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f0e77-c798-4f91-a54d-8ceddbccc571 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.329299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.330040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.330330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.330701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.331282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.333198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.422477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 987/1964] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:10.423911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.453996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 978/1965] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:10.456607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.473466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:10.484815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d331843-5c0b-4867-acf0-0c0334833114 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.599885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c08900-9eef-43bf-8df9-ef0b2df7c020 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.615239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.616076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.616795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.626207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.627413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.629663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.639410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.640142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.640491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.640832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.641570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.643634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.675326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:10.676522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 979/1966] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:10.677512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.736188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 988/1967] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/projects => generated 383 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:10.736188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.791133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8fc89-9e28-46bb-9378-183e8796c8b8 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.840398 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486381c-4a74-4090-b96c-6ee214cc1242 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.846348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.847204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.847792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.848220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.849641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.851490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.867218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.868021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.868376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.869114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.874009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 989/1968] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:10.875038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.895461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8fd8a-2bdc-4a59-82d2-8daac5d41813 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.904962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.906464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.906926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.907395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.908407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.910468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.938755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 990/1969] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:10 2022] POST /identity/v3/users => generated 292 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:10.939600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.941266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:10.954479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1c78b-a8df-4959-a6f9-63d7a1fdca47 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:10.955250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.957161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.962592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:10.963559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:10.963849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:10.964201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] PATH_INFO: `/v3/users/efd92fe270e1429b87d12f9bf093dec1` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:10.965430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd92fe270e1429b87d12f9bf093dec1)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:10.968423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:10.972400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 980/1970] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:10.973552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:10.982550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:10.990012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e41145-e92d-4cda-a88c-3fa7e97b61e1 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.002513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.003770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.004236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.004865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] PATH_INFO: `/v3/projects/48f444b71c5543b18224dd5174636a53/users/a65cbc8e25e14a9b8a53db8c1cefdd18/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.028261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f444b71c5543b18224dd5174636a53, user_id=a65cbc8e25e14a9b8a53db8c1cefdd18, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.036118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.063405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 981/1971] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:10 2022] PUT /identity/v3/projects/48f444b71c5543b18224dd5174636a53/users/a65cbc8e25e14a9b8a53db8c1cefdd18/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:11.069771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.077127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd92fe270e1429b87d12f9bf093dec1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:11.078434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd92fe270e1429b87d12f9bf093dec1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:11.095753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd92fe270e1429b87d12f9bf093dec1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:11.144051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd92fe270e1429b87d12f9bf093dec1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:11.152140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd92fe270e1429b87d12f9bf093dec1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:11.161752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 991/1972] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:10 2022] DELETE /identity/v3/users/efd92fe270e1429b87d12f9bf093dec1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:11.163783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.165623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:11.244493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:11.247816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd2ff-2130-4b3e-a19a-f5a0ae5428ea admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.288649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a8984-883c-49f1-b62e-c1a26ccf230e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.294560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.295484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.295884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.296338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.297146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.299692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072dea-b356-4539-af4e-6e5bcbe2b48d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.316488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 992/1973] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:11.316488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.316488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.316488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.355804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:11.357105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:11.368103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 993/1974] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:11.368876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.391939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120daf2-ef70-4d2f-89b9-4356fb44b9fb None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.401718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.403594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.405772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.406220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] PATH_INFO: `/v3/projects/48f444b71c5543b18224dd5174636a53/users/a65cbc8e25e14a9b8a53db8c1cefdd18/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.411514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48f444b71c5543b18224dd5174636a53, user_id=a65cbc8e25e14a9b8a53db8c1cefdd18, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.419486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.424168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.424856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.425197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.425526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.427291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.431641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.455100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 982/1975] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:11.460450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.460828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 994/1976] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:11 2022] PUT /identity/v3/projects/48f444b71c5543b18224dd5174636a53/users/a65cbc8e25e14a9b8a53db8c1cefdd18/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:11.461771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.463003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:11.463624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:11.581353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc66d-fb96-4d57-96d0-b9e0aa34a099 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.589577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25576e-3ceb-4d57-b58e-3065b4bbc340 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.592491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.592654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.592923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.593197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.601997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.603835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.614582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 995/1977] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:11.618801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.631560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347f85f-e165-4a39-9b7e-ca09e30415ae admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:11.637233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.638330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.638753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.639100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] PATH_INFO: `/v3/projects/52547ce793824d7c8de38dbd606bd435/users/2d3cb696fd0e4c678bc0271fecf7c334/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.657273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:11.659697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52547ce793824d7c8de38dbd606bd435, user_id=2d3cb696fd0e4c678bc0271fecf7c334, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.668465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9e5b6-55dc-417b-8991-3455c6f328f5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.677731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:11.678533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.679109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.679432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.690975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 996/1978] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:11 2022] PUT /identity/v3/projects/52547ce793824d7c8de38dbd606bd435/users/2d3cb696fd0e4c678bc0271fecf7c334/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:11.694803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:11.696394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:11.708089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 983/1979] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:11.713876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:11.714110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:11.714528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:11.768151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] MFA Rules not processed for user `2d3cb696fd0e4c678bc0271fecf7c334`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:11.769780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:11.875125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 997/1980] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:11.879479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:11.896635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4877a-03c7-4271-9c17-7b1e0dbe91c7 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.009339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.009731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.010046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.010320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.011053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.014875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.096001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 984/1981] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:11 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:12.101544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:12.135598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259e11-bc41-495a-b6ae-5ba7e380646e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.164286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:12.251742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.252419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.252732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.253074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.253754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.258371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.307789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 998/1982] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:12 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:12.311505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:12.322605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97abbf-b673-4c22-a754-c8172ec25ede admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.327971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.328688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.329100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.329601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.330635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.332418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.354780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 985/1983] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:12.359360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:12.399540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d974-7099-4473-8e34-a4677cb8dab3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.417126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.419079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.419311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.419572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] PATH_INFO: `/v3/projects/76efe5f5d14f48d3a2a56e237819596a/users/cb55af5513fa453f837990ded6eb1582/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.491344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76efe5f5d14f48d3a2a56e237819596a, user_id=cb55af5513fa453f837990ded6eb1582, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.496182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.530785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:12.532727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 999/1984] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:12 2022] PUT /identity/v3/projects/76efe5f5d14f48d3a2a56e237819596a/users/cb55af5513fa453f837990ded6eb1582/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:12.557474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1ec-0bfe-469d-a047-a6c013d29f05 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.774170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.775234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.778706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.780055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.780954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.782822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3f79-024c-47ac-b7ab-a755e8256f29 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.841419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 986/1985] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:12.845467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:12.865159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd98e5-3d5b-4603-a0b1-e98c927d1668 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:12.896548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:12.897475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.897979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.898436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] PATH_INFO: `/v3/projects/76efe5f5d14f48d3a2a56e237819596a/users/cb55af5513fa453f837990ded6eb1582/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:12.905524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76efe5f5d14f48d3a2a56e237819596a, user_id=cb55af5513fa453f837990ded6eb1582, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:12.915216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:12.958657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1000/1986] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:12 2022] PUT /identity/v3/projects/76efe5f5d14f48d3a2a56e237819596a/users/cb55af5513fa453f837990ded6eb1582/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:12.961450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:12.961996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:12.962183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:13.023703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] MFA Rules not processed for user `cb55af5513fa453f837990ded6eb1582`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:13.027427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:13.049853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:13.052500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:13.196112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:13.231999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a49d-73ca-46b6-8613-ac03a38dc783 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:13.256070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:13.256435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:13.256930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:13.257505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] PATH_INFO: `/v3/projects/0ca2bad3de4e4e6f8e2a30053c3989b1` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:13.259606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca2bad3de4e4e6f8e2a30053c3989b1)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:13.262161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:13.274000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 987/1987] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:13.275112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:13.285222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:13.379789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca2bad3de4e4e6f8e2a30053c3989b1', 'request_id': 'req-29520c45-0991-41ed-b62d-9ac358fea821'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.397103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca2bad3de4e4e6f8e2a30053c3989b1', 'request_id': 'req-29520c45-0991-41ed-b62d-9ac358fea821'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.414443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca2bad3de4e4e6f8e2a30053c3989b1', 'request_id': 'req-29520c45-0991-41ed-b62d-9ac358fea821'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.448328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0c59d-8353-43c2-8432-420db4d28e5e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:13.455137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] The token cache is being invalidate because project 0ca2bad3de4e4e6f8e2a30053c3989b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:13.455449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.469262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.482737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1001/1988] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:13 2022] DELETE /identity/v3/projects/0ca2bad3de4e4e6f8e2a30053c3989b1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:13.485748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:13.490746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:13.502256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:13.593768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:13.594873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:13.598570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:13.598669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:13.621063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:13.681104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29520c45-0991-41ed-b62d-9ac358fea821 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:13.691877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:13.692548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:13.692832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:13.693110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] PATH_INFO: `/v3/users/d585c0b792c04a0c92c54bc087e45563` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:13.703163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d585c0b792c04a0c92c54bc087e45563)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:13.705129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:13.823794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:13.827220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:13.840898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd585c0b792c04a0c92c54bc087e45563'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.845705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd585c0b792c04a0c92c54bc087e45563'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.856989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd585c0b792c04a0c92c54bc087e45563'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.873924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 988/1989] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:13.913093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd585c0b792c04a0c92c54bc087e45563'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.945054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd585c0b792c04a0c92c54bc087e45563'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:13.956980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1002/1990] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:13 2022] DELETE /identity/v3/users/d585c0b792c04a0c92c54bc087e45563 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:14.070923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:14.072938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:14.094318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1982f-86f3-4f64-9a27-f0f35ce1a9c9 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:14.226189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:14.246476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:14.250801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:14.251288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:14.251733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:14.253843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:14.431909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:14.433901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:14.459060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 989/1991] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:15.047510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:15.049099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.230321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab533a-fe51-4e77-8687-ac49eaadddaa admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:15.248287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:15.249025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.249315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.249653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] PATH_INFO: `/v3/users/2d3cb696fd0e4c678bc0271fecf7c334` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.253366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3cb696fd0e4c678bc0271fecf7c334)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:15.257010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:15.314451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:15.316107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.327856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:15.387763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3cb696fd0e4c678bc0271fecf7c334'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.390011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3cb696fd0e4c678bc0271fecf7c334'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.391976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b4d0-5da2-4b55-b709-0ff78200ca6b service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:15.403967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3cb696fd0e4c678bc0271fecf7c334'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.420811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:15.423315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.430168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.430356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] PATH_INFO: `/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.433665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4f78d24aee4c0980aa0e9cf88a9b7c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:15.443133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3cb696fd0e4c678bc0271fecf7c334'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.444179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:15.458534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bfdb068-0588-4f18-b410-890ff61d6d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3cb696fd0e4c678bc0271fecf7c334'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.472939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1003/1992] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:15 2022] DELETE /identity/v3/users/2d3cb696fd0e4c678bc0271fecf7c334 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:15.474681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7cce-3837-4b75-9d2f-649c6b113d9a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.474845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7cce-3837-4b75-9d2f-649c6b113d9a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.475123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7cce-3837-4b75-9d2f-649c6b113d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.550612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-74ad7cce-3837-4b75-9d2f-649c6b113d9a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:15.551861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-74ad7cce-3837-4b75-9d2f-649c6b113d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.572999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4f78d24aee4c0980aa0e9cf88a9b7c', 'request_id': 'req-885e10b1-411c-4cf7-8c74-bf06d5987b70'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.589432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4f78d24aee4c0980aa0e9cf88a9b7c', 'request_id': 'req-885e10b1-411c-4cf7-8c74-bf06d5987b70'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.605945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4f78d24aee4c0980aa0e9cf88a9b7c', 'request_id': 'req-885e10b1-411c-4cf7-8c74-bf06d5987b70'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.646231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] The token cache is being invalidate because project 3b4f78d24aee4c0980aa0e9cf88a9b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:15.647898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.650191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.653403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 990/1993] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:15 2022] DELETE /identity/v3/projects/3b4f78d24aee4c0980aa0e9cf88a9b7c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:15.655501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:15.661355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.717066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1004/1994] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:15.718268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4df4f8-1e2e-4e4b-8863-c43a312e2744 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.718557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4df4f8-1e2e-4e4b-8863-c43a312e2744 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.718832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4df4f8-1e2e-4e4b-8863-c43a312e2744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.813283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ef4df4f8-1e2e-4e4b-8863-c43a312e2744 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:15.814929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ef4df4f8-1e2e-4e4b-8863-c43a312e2744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.824962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1005/1995] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:15.851720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.852383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.852809 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.855377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e10b1-411c-4cf7-8c74-bf06d5987b70 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:15.861994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:15.863512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.864509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.865059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] PATH_INFO: `/v3/users/35ccc311ff6d4df1ab19ff2e0712bdea` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.866668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ccc311ff6d4df1ab19ff2e0712bdea)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:15.869161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:15.904448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:15.908758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:15.925175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:15.931042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1006/1996] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:15.936689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:15.962636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3025b-fcd5-4e4c-9c4b-fc67671a6611 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:15.976173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ccc311ff6d4df1ab19ff2e0712bdea'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:15.979501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:15.985116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:15.985508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:15.985794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:15.986934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:15.987247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:15.988938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.000560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ccc311ff6d4df1ab19ff2e0712bdea'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:16.001397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ccc311ff6d4df1ab19ff2e0712bdea'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:16.016509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1007/1997] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:16.016509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.017333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.018545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ccc311ff6d4df1ab19ff2e0712bdea'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:16.019292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ccc311ff6d4df1ab19ff2e0712bdea'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:16.039874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 991/1998] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:15 2022] DELETE /identity/v3/users/35ccc311ff6d4df1ab19ff2e0712bdea => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:16.044328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.049576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.128621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddcc26-2c2e-4940-a0fa-8021d4cfb3fe admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.144760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36129aa-a20d-4b51-93c6-01bb6d7bd86a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.150176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.151163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.151716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.152098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.152968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.155193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b5cb-7e89-4ff2-ad93-a32cb67a2215 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.167017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:16.202911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 992/1999] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:16 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:16.205970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.206243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.208648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.229963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.230619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.230904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.231178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.237791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.253329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:16.253811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.255294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.255413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.266801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1008/2000] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:16.267868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.278013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7e4-b9f0-4275-8b27-13ccb2cf0d2c service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.397605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.399269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.399543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.401302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.402736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.405318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.409595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 993/2001] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:16.420760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.422853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.475091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1009/2002] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:16 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:16.480492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.518177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76aadc-ba5b-465e-a90f-449929ec8de5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.518705 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:16.538896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.541187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.541484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.542590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.543827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.551250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.596540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1010/2003] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:16.598271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.625065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa2f99-1c34-49cc-aa36-2f3412b2d969 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.638154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.639376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e8b29-f936-40ec-87a8-8c65d4c49eee admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.653747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.654700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.655010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.662517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.663519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.663806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.667050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.667857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.671245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.680843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.682426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b1a22-cf5e-44e9-8b7c-2defa1d1864a service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.697927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1011/2004] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:16.702631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:16.714358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 994/2005] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:16.725466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720f4a-65f3-4617-a9d3-07a9750e8518 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:16.731168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:16.731972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.733376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.733588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] PATH_INFO: `/v3/projects/930e382d20b041dd9a924f9af67ca44f/users/3b3700729508491fb8ee3817fcc933b6/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.753187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930e382d20b041dd9a924f9af67ca44f, user_id=3b3700729508491fb8ee3817fcc933b6, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:16.756416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:16.776361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1012/2006] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:16 2022] PUT /identity/v3/projects/930e382d20b041dd9a924f9af67ca44f/users/3b3700729508491fb8ee3817fcc933b6/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:16.788831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:16.788831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:16.789090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:16.832731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] MFA Rules not processed for user `3b3700729508491fb8ee3817fcc933b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:16.834043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:16.937934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 995/2007] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:16.942516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.072963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc41c-083b-4735-9119-4e1d494aa2a8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.082489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.083375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.083875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.084360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.085246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.087559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.146360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.148714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:17.156839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1013/2008] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:16 2022] POST /identity/v3/projects => generated 369 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:17.157967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.163496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:17.181693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb274d6-54a6-4810-9987-3e9e76096592 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.318686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565d4c-deb3-4aad-98ce-ed2532d19090 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.328242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.328740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.329026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.329300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] PATH_INFO: `/v3/projects/5120403609804c31b7c42db72bd81502` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.331108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5120403609804c31b7c42db72bd81502)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.335068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.336348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.336629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.336899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.337581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.338277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.341652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.401119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1014/2009] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:17 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:17.402505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.421254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa20ed-a90c-4cd4-8c61-6b6f210d4d12 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.445410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.446244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.446695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.447040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.447881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.448381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5120403609804c31b7c42db72bd81502', 'request_id': 'req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:17.449770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.456493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5120403609804c31b7c42db72bd81502', 'request_id': 'req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:17.464295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1015/2010] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:17.469424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5120403609804c31b7c42db72bd81502', 'request_id': 'req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:17.471564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:17.481082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.495991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] The token cache is being invalidate because project 5120403609804c31b7c42db72bd81502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:17.495991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:17.496301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3fc9a787-7c96-4724-a3d3-2148e9f4af0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:17.502606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 996/2011] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:17 2022] DELETE /identity/v3/projects/5120403609804c31b7c42db72bd81502 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:17.509786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4d97e-6de6-454d-b7dd-7b6eac1d8497 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.512866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d4ce-feef-4ad1-b89c-61aaa30f660a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.515747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4d97e-6de6-454d-b7dd-7b6eac1d8497 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.516373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4d97e-6de6-454d-b7dd-7b6eac1d8497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.560833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-37d4d97e-6de6-454d-b7dd-7b6eac1d8497 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:17.563003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-37d4d97e-6de6-454d-b7dd-7b6eac1d8497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:17.622141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.623761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.624238 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.624692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] PATH_INFO: `/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f/users/e881752e6efa4bd7b224d13ea29c9a54/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.643970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9c6e4e2b6d4af597ae39e0fefef71f, user_id=e881752e6efa4bd7b224d13ea29c9a54, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.647889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.676416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 997/2012] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:17.678467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.678877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.679282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.701664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1016/2013] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:17 2022] PUT /identity/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f/users/e881752e6efa4bd7b224d13ea29c9a54/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:17.704890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.708024 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:17.726377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:17.729054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:17.853920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81453993-c625-459c-937f-8abf2c8620d5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.886335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.886986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.887269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.887543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.888675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.893559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.901036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 998/2014] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:17.903069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.907868 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:17.914626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad00a8-7145-439f-9b9a-4adbbd9f52f0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.920124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.920828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.921181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.921557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.922456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:17.923044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1017/2015] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:17.923280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.923912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.925230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.938102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f510d07-75d1-4064-b2ef-1c54e29fe5ab admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.944344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 999/2016] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:51:17.944520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:17.944520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.944520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.944733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] PATH_INFO: `/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f/users/e881752e6efa4bd7b224d13ea29c9a54/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:17.946482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:17.947301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9c6e4e2b6d4af597ae39e0fefef71f, user_id=e881752e6efa4bd7b224d13ea29c9a54, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:17.950418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a103f-c8ce-49dc-84ca-282169c7b753 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:17.962430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08747bb-32d1-4bdc-afdf-2d8db771a0dc admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:17.964263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1018/2017] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:17 2022] PUT /identity/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f/users/e881752e6efa4bd7b224d13ea29c9a54/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:17.973538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:17.973844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:17.974130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.024043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] MFA Rules not processed for user `e881752e6efa4bd7b224d13ea29c9a54`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:18.026865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.093137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.094778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.095589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.096098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.097911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.101167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.146315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1019/2018] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:18.147557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.149303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.165927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1000/2019] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:17 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:18.170467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.171573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f0d3-95e2-4b94-9ffe-e20e424501a4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.180785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:18.281688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.282604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.282938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.283215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] PATH_INFO: `/v3/projects/52547ce793824d7c8de38dbd606bd435` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.284795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52547ce793824d7c8de38dbd606bd435)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.287010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.291691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfc4b3-b4d2-4702-ac90-6c3d09c4e1c1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.300229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.302070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.302595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.303358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.305081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.309535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.355869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1001/2020] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:18 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:18.361486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.365139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.366101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52547ce793824d7c8de38dbd606bd435', 'request_id': 'req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.374944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52547ce793824d7c8de38dbd606bd435', 'request_id': 'req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.384655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52547ce793824d7c8de38dbd606bd435', 'request_id': 'req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.412067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] The token cache is being invalidate because project 52547ce793824d7c8de38dbd606bd435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:18.412402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.413059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.419052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1020/2021] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:18 2022] DELETE /identity/v3/projects/52547ce793824d7c8de38dbd606bd435 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:18.419052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.419052 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.424325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:18.494564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e92031-023f-412a-b13b-249899ad4c75 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.510242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa903ea-2c6c-4e88-8d7c-84b179b863d8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.517625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.517625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.517625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.518121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.519834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.521987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.540972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1021/2022] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:18.540972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.542124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.561301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cef1bd-0bbb-4283-91d1-69f39d96cd68 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.567551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.567789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.568018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.568279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.568383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.568551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] PATH_INFO: `/v3/users/cb55af5513fa453f837990ded6eb1582` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.568655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.569457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.569752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb55af5513fa453f837990ded6eb1582)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.571494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.572011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.645516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb55af5513fa453f837990ded6eb1582'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.646173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb55af5513fa453f837990ded6eb1582'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.657277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb55af5513fa453f837990ded6eb1582'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.669060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.670888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.676413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:18.677377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb55af5513fa453f837990ded6eb1582'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.689933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb55af5513fa453f837990ded6eb1582'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:18.692523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1002/2023] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:18 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:18.693047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.697405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1022/2024] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:18 2022] DELETE /identity/v3/users/cb55af5513fa453f837990ded6eb1582 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:18.714375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd8be4-533e-4092-9b69-37b2cd58ab7c service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.852512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.853517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.853796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.854138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.858064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.863337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b143d8-8ab6-4e18-b5c4-a1a6c679bdcb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.914930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:18.915530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:18.917322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1003/2025] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:18.933897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3181f9-420f-48d6-8b4f-fe7e18483ecb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:18.940843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:18.941908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:18.942169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:18.942453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] PATH_INFO: `/v3/projects/c62ccb69bf5240f6a042708137e9bef6/users/71af650c5ddd4622a2b8ae2b9bc385bf/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:18.963191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62ccb69bf5240f6a042708137e9bef6, user_id=71af650c5ddd4622a2b8ae2b9bc385bf, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:18.968998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:18.993003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1023/2026] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:18 2022] PUT /identity/v3/projects/c62ccb69bf5240f6a042708137e9bef6/users/71af650c5ddd4622a2b8ae2b9bc385bf/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:19.003854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:19.004162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:19.004416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:19.039360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] MFA Rules not processed for user `71af650c5ddd4622a2b8ae2b9bc385bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:19.041784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:19.158347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1004/2027] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:19.170209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:19.172406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:19.338843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:19.403885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643773c-aaea-4b05-95fa-606d07e729fb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:19.416638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:19.418113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:19.418609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:19.419660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:19.442327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:19.453134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3201e-9879-4737-bd3a-a5c43c12cf4d service glance] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:19.478958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1024/2028] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:20.374009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:20.375606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:20.541295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b1f8-264f-4f8e-a99f-4fd61170dde9 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:20.551716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:20.551888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:20.553789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:20.557012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] PATH_INFO: `/v3/projects/76efe5f5d14f48d3a2a56e237819596a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:20.558373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76efe5f5d14f48d3a2a56e237819596a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:20.564998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:20.643786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:20.670055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76efe5f5d14f48d3a2a56e237819596a', 'request_id': 'req-2d26eb44-6429-4219-95c4-fe4a837b6a24'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:20.682671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76efe5f5d14f48d3a2a56e237819596a', 'request_id': 'req-2d26eb44-6429-4219-95c4-fe4a837b6a24'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:20.698242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76efe5f5d14f48d3a2a56e237819596a', 'request_id': 'req-2d26eb44-6429-4219-95c4-fe4a837b6a24'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:20.727257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] The token cache is being invalidate because project 76efe5f5d14f48d3a2a56e237819596a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:20.727554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:20.728215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2d26eb44-6429-4219-95c4-fe4a837b6a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:20.732192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1005/2029] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:20 2022] DELETE /identity/v3/projects/76efe5f5d14f48d3a2a56e237819596a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:20.737170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:20.737657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:20.737733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:20.777036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:20.777953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:20.942380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1025/2030] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:20.952595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738773-953d-4082-8f7c-605941974227 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:20.953776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738773-953d-4082-8f7c-605941974227 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:20.954241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc738773-953d-4082-8f7c-605941974227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.019453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cc738773-953d-4082-8f7c-605941974227 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:21.021013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cc738773-953d-4082-8f7c-605941974227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:21.052877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1006/2031] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:21.055619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.066592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd1363-dd1c-4282-b31a-0706c888295e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.094538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.095189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.095740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.096081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.096814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:21.097122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.098855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.134488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1026/2032] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:21.143660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc738773-953d-4082-8f7c-605941974227 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.174716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc738773-953d-4082-8f7c-605941974227 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.196137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.196137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.196137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.196137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.196612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.196612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.277911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1007/2033] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:21 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:21.284538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.298077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f1615-c5e7-4524-b2e7-a864bc3e721b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.490017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.490738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.490973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.491240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.495166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.497237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.547251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.553390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:21.554788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1027/2034] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:21 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:21.558841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.578639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65faf46b-e936-4c3e-8ce3-bfa86bcf5712 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.596343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.598789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.599967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.601159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.604211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.608063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.619217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a30-627b-4d55-8ac1-923287bb3586 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.642584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1028/2035] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:21.645413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.646379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.650585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.652492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.653044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] PATH_INFO: `/v3/users/3b3700729508491fb8ee3817fcc933b6` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.655997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3700729508491fb8ee3817fcc933b6)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.658243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.662630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc9559-bf24-4763-8771-97a75a89ea61 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.679155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.679155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.679155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.679155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.679750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.683887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.717088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1029/2036] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:21.722706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:21.744894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666bd6f-4bca-445b-ac3e-7dcbaa1dd52d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:21.767301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:21.768313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.769370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.769801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] PATH_INFO: `/v3/projects/db49de5691974be28efa13dd4e0d2d2b/users/d4265c18590c41ec9e8c43053104da12/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.789920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3700729508491fb8ee3817fcc933b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:21.807925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3700729508491fb8ee3817fcc933b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:21.809288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3700729508491fb8ee3817fcc933b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:21.812329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db49de5691974be28efa13dd4e0d2d2b, user_id=d4265c18590c41ec9e8c43053104da12, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:21.815706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5888d9-96b3-4c7c-852c-afe9c8b2c5d7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:21.833598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:21.857878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3700729508491fb8ee3817fcc933b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:21.858121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1030/2037] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:21 2022] PUT /identity/v3/projects/db49de5691974be28efa13dd4e0d2d2b/users/d4265c18590c41ec9e8c43053104da12/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:21.858309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-2270fcdc-3a7c-476e-808f-891cc5080d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3700729508491fb8ee3817fcc933b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:21.869423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.870228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.870779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.905234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1008/2038] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:21 2022] DELETE /identity/v3/users/3b3700729508491fb8ee3817fcc933b6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:21.910639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:21.910934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:21.911202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:21.959372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] MFA Rules not processed for user `d4265c18590c41ec9e8c43053104da12`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:21.961099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:21.963890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:21.965129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:22.142765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:22.168508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1031/2039] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:22.171493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1009/2040] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:22.190405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:22.192087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:22.209190 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:22.210783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:22.350701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3b96-fc63-41c2-be4e-3dcf5c84c774 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:22.355215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714898aa-5e5b-4fc2-b0ac-ee9777b64f8e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:22.356105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:22.356805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:22.357197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:22.357486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:22.368307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:22.369192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:22.369681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:22.369812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:22.370273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:22.371685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97761a-90f2-43b8-9bc0-97d89c3abcd2 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:22.392882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1010/2041] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:22.394115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:22.395789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:22.415031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1032/2042] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:23.179563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:23.180482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:23.180877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:23.227684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] MFA Rules not processed for user `a65cbc8e25e14a9b8a53db8c1cefdd18`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:23.229028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:23.290020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:23.292344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:23.344766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1eaf-eff4-499c-8e17-cb2e8183ad72 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:23.353392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:23.368326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:23.368886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:23.370356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] PATH_INFO: `/v3/projects/930e382d20b041dd9a924f9af67ca44f` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:23.371208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1011/2043] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:23.372827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=930e382d20b041dd9a924f9af67ca44f)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:23.375708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:23.394241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:23.395986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:23.438406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930e382d20b041dd9a924f9af67ca44f', 'request_id': 'req-ad0453f3-fffe-4725-8551-a1b31b5a73d4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.445687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930e382d20b041dd9a924f9af67ca44f', 'request_id': 'req-ad0453f3-fffe-4725-8551-a1b31b5a73d4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.455046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930e382d20b041dd9a924f9af67ca44f', 'request_id': 'req-ad0453f3-fffe-4725-8551-a1b31b5a73d4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.469306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] The token cache is being invalidate because project 930e382d20b041dd9a924f9af67ca44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:23.469599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.470549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.472126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1033/2044] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:23 2022] DELETE /identity/v3/projects/930e382d20b041dd9a924f9af67ca44f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:23.476343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:23.478715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:23.502493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d60de-b704-47d0-af01-4dc51287f9ca None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:23.507917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:23.528825 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:23.592358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:23.593071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:23.593665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:23.594103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:23.597024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:23.604650 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0453f3-fffe-4725-8551-a1b31b5a73d4 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:23.615512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:23.617019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:23.617406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:23.617778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] PATH_INFO: `/v3/users/e881752e6efa4bd7b224d13ea29c9a54` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:23.633135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e881752e6efa4bd7b224d13ea29c9a54)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:23.636157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:23.704419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:23.710370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:23.724149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1012/2045] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:23.773887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e881752e6efa4bd7b224d13ea29c9a54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.774602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e881752e6efa4bd7b224d13ea29c9a54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.784018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e881752e6efa4bd7b224d13ea29c9a54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.804847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e881752e6efa4bd7b224d13ea29c9a54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.816607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30cde861-38e9-4d36-afb7-40f841c13f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e881752e6efa4bd7b224d13ea29c9a54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:23.830981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1034/2046] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:23 2022] DELETE /identity/v3/users/e881752e6efa4bd7b224d13ea29c9a54 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:24.282048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:24.295741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b8138-21fc-4713-bb40-14aa748f1886 service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:24.427307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:24.428053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.428545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.428884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] PATH_INFO: `/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.430970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9c6e4e2b6d4af597ae39e0fefef71f)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:24.433479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:24.532259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9c6e4e2b6d4af597ae39e0fefef71f', 'request_id': 'req-a6946728-e424-4261-837c-d9757803ad7f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:24.542513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9c6e4e2b6d4af597ae39e0fefef71f', 'request_id': 'req-a6946728-e424-4261-837c-d9757803ad7f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:24.552816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9c6e4e2b6d4af597ae39e0fefef71f', 'request_id': 'req-a6946728-e424-4261-837c-d9757803ad7f'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:24.555441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:24.580934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] The token cache is being invalidate because project ea9c6e4e2b6d4af597ae39e0fefef71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:24.581400 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:24.582103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-a6946728-e424-4261-837c-d9757803ad7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:24.583643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1013/2047] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:24 2022] DELETE /identity/v3/projects/ea9c6e4e2b6d4af597ae39e0fefef71f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:24.588468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.588956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.589146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.611831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:24.613288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:24.697893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1035/2048] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:24.706061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.706353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.706633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.730392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:24.731625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:24.740215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1014/2049] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:24.744589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:24.759444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0373b-d5fc-45bd-b1d7-a4889f324cfd None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:24.764913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:24.765542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.765852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.766241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.766953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:24.767265 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:24.768880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:24.781451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1036/2050] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:24.786010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:24.797094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6b808-7e7a-457e-a2a9-ac3c59529291 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:24.803124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:24.803886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.804169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.804474 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.805262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:24.807219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:24.841857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1015/2051] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:24 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:24.846084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:24.858619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98c509-7cd9-4736-b590-be11b180d94c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:24.879654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:24.933453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:24.934122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.934407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.934683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.935415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:24.937357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:24.962424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1037/2052] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:24 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:24.963732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:24.975488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d75ea-9493-4c43-8d27-57fdccbc97e2 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:24.981785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:24.983179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:24.983179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:24.983179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:24.983820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:24.986745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.003073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1016/2053] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:25.010191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.037231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d201527-419c-4e1c-b0bf-0846bbea9313 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.046432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.047160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.047606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.047890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.049321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.051148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53423e4-e1f0-4137-9b7a-ed43fdd7368a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.063002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1038/2054] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:25.069546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.084303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19fc0-bc09-404b-9634-370816020b4a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.089363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.090112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.090394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.090655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] PATH_INFO: `/v3/projects/4bc8c604677141cc885fc525d7349462/users/80b80e9a74474dff89272017f3a1972e/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.106296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.106612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.106870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.117221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc8c604677141cc885fc525d7349462, user_id=80b80e9a74474dff89272017f3a1972e, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.120329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48add165-0956-4963-a2c0-a464546a8823 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.137684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] MFA Rules not processed for user `a1c132b2fdc14e7ca4ffb18109a0fa0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:25.139037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:25.141534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1017/2055] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:25 2022] PUT /identity/v3/projects/4bc8c604677141cc885fc525d7349462/users/80b80e9a74474dff89272017f3a1972e/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:25.155090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.155445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.155659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.191085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] MFA Rules not processed for user `80b80e9a74474dff89272017f3a1972e`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:25.192697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:25.230013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1039/2056] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:25.305237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.307676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1018/2057] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:25.406723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c8ac1-affa-4838-b05b-17f5b4fa91a9 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.414747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.415469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.416039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.416530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.417200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.419209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.457000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1040/2058] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:25 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:25.458181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.469560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26d10a-3036-4d60-b7e4-f7e20f5e29c7 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.545197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.546349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.546852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.547510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.548722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.551189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.585137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1019/2059] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:25 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:25.587183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.605414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd974ca2-f7c3-4b26-abc3-1c918831409a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.611250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.612116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.612608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.613167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.614175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.615895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.628994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1041/2060] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:25.633109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.646831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238aff7-443c-4b56-86c1-2ed819a1285f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.666159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.667418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.668502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.669139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] PATH_INFO: `/v3/projects/988b819b7b66403dab93a27e35deaffc/users/94bd17b3abeb4c57873c0d37f504daae/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.674436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:25.696646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988b819b7b66403dab93a27e35deaffc, user_id=94bd17b3abeb4c57873c0d37f504daae, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.700240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.728523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1020/2061] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:25 2022] PUT /identity/v3/projects/988b819b7b66403dab93a27e35deaffc/users/94bd17b3abeb4c57873c0d37f504daae/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:25.729609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.747681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bc3d9-f715-4333-9e70-2e06e2760ceb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.777847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:25.851359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.853188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.853480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.853765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.854694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.859790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121b1b6-3feb-4f20-8d05-d05ec5ae7ac2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.874504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1042/2062] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:25.878674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:25.898679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be2295-5fbe-4b33-a87e-301b8a611097 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:25.903374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:25.904649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.904936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.905341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] PATH_INFO: `/v3/projects/988b819b7b66403dab93a27e35deaffc/users/94bd17b3abeb4c57873c0d37f504daae/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.907850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] RBAC: Authorizing `identity:create_grant(project_id=988b819b7b66403dab93a27e35deaffc, user_id=94bd17b3abeb4c57873c0d37f504daae, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:25.911288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:25.925055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1021/2063] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:25 2022] PUT /identity/v3/projects/988b819b7b66403dab93a27e35deaffc/users/94bd17b3abeb4c57873c0d37f504daae/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:25.936420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:25.936709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:25.936988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:25.969420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] MFA Rules not processed for user `94bd17b3abeb4c57873c0d37f504daae`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:25.971106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:26.044091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1043/2064] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:26.055489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:26.057213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:26.117717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa59f9-3d3b-446b-b648-419db8d7229a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:26.122463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:26.125757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:26.126113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:26.126387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:26.136080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:26.137541 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e236-fab7-43c3-b7a1-b678db329a37 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:26.147304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1022/2065] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:26.214307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:26.281750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e1358-f763-4200-923c-1ec15164df01 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:26.286482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:26.287278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:26.287560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:26.287831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] PATH_INFO: `/v3/users/80b80e9a74474dff89272017f3a1972e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:26.289017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b80e9a74474dff89272017f3a1972e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:26.290963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:26.349500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:26.352576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b80e9a74474dff89272017f3a1972e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:26.353470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b80e9a74474dff89272017f3a1972e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:26.363789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b80e9a74474dff89272017f3a1972e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:26.376978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b80e9a74474dff89272017f3a1972e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:26.385842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b80e9a74474dff89272017f3a1972e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:26.393969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1044/2066] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:26 2022] DELETE /identity/v3/users/80b80e9a74474dff89272017f3a1972e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:26.398816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:26.399660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:26.399934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:26.428768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:26.430805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:26.508941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1023/2067] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:26.519142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:26.520589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:26.593743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24230e-f633-411a-b34c-f7b88da02645 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:26.597757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:26.598410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:26.598696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:26.598972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:26.608304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:26.609727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:26.619245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1045/2068] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:26.927956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:26.929485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:26.946199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e889d-543f-4ca2-8bca-27371e092f48 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:26.953434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:26.954155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:26.954441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:26.954714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] PATH_INFO: `/v3/projects/4bc8c604677141cc885fc525d7349462` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:26.955907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc8c604677141cc885fc525d7349462)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:26.958937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:27.011213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc8c604677141cc885fc525d7349462', 'request_id': 'req-cbdaa101-f8c8-47b6-a4e6-52996608df1d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.018040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc8c604677141cc885fc525d7349462', 'request_id': 'req-cbdaa101-f8c8-47b6-a4e6-52996608df1d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.026468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc8c604677141cc885fc525d7349462', 'request_id': 'req-cbdaa101-f8c8-47b6-a4e6-52996608df1d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.031653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:27.039454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] The token cache is being invalidate because project 4bc8c604677141cc885fc525d7349462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:27.040323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.041301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.043267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1024/2069] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:26 2022] DELETE /identity/v3/projects/4bc8c604677141cc885fc525d7349462 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:27.047517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:27.058245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dd95-80a9-4cf5-9a36-3bb9d2ced745 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:27.129997 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:27.131007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:27.131292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:27.131566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] PATH_INFO: `/v3/users/94bd17b3abeb4c57873c0d37f504daae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:27.132980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bd17b3abeb4c57873c0d37f504daae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:27.135013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:27.189955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bd17b3abeb4c57873c0d37f504daae'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.190541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bd17b3abeb4c57873c0d37f504daae'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.199033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bd17b3abeb4c57873c0d37f504daae'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.208365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:27.214749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bd17b3abeb4c57873c0d37f504daae'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.225752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4e512dc5-5dcd-4862-a655-995a137f11dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bd17b3abeb4c57873c0d37f504daae'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.234406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1046/2070] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:27 2022] DELETE /identity/v3/users/94bd17b3abeb4c57873c0d37f504daae => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:27.527033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:27.530731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:27.549975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaa101-f8c8-47b6-a4e6-52996608df1d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:27.647722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:27.648496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:27.648667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:27.648920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] PATH_INFO: `/v3/projects/988b819b7b66403dab93a27e35deaffc` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:27.651536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988b819b7b66403dab93a27e35deaffc)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:27.653949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:27.715169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988b819b7b66403dab93a27e35deaffc', 'request_id': 'req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.725987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988b819b7b66403dab93a27e35deaffc', 'request_id': 'req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.735731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988b819b7b66403dab93a27e35deaffc', 'request_id': 'req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.750796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] The token cache is being invalidate because project 988b819b7b66403dab93a27e35deaffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:27.751092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.752100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c0f6e0cc-11c9-4621-b80e-27b7f4e73342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:27.755362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1025/2071] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:27 2022] DELETE /identity/v3/projects/988b819b7b66403dab93a27e35deaffc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:27.760266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:27.760758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:27.760843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:27.791100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:27.792426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:27.868407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1047/2072] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:27.876009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:27.876572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:27.876943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:27.928499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:27.929759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:27.941553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1026/2073] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:27.942748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:27.954693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c9919-a178-4b80-8389-78cd79ecf6a3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:27.959648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:27.960447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:27.961092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:27.961367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:27.962286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:27.962610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:27.966006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:27.986406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1048/2074] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:27.987530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:27.998177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce974-9a07-4d64-9b43-08091ed07387 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.003894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.004554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.004991 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.005245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.005977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.007936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.048505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1027/2075] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:27 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:28.053181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.063655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ef9fb-dbea-4544-b880-75db24ea5726 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.134651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.135310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.135594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.135872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.136545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.139137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.169938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1049/2076] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:28 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:28.171449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.182038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803449d-2495-4ac2-a229-fee6aa0f0e04 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.189725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.190468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.191468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.191468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.192015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.193954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.213184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1028/2077] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:28.217442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.230678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a1226-f25b-4f92-8150-7711daff857b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.239385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.240088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.240378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.240651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.241355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.243034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ace21-9b7a-4a2a-ac45-7d05f4e8320a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.261969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1050/2078] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:28.263713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.277880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52544380-e215-4d93-99a2-e5d153683621 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.296418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.298230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.298561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.299544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] PATH_INFO: `/v3/projects/44eac1bf35034b85b7404b4c3fd71140/users/ba51ce92ff494accbb420e1f03f143a0/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.330203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=44eac1bf35034b85b7404b4c3fd71140, user_id=ba51ce92ff494accbb420e1f03f143a0, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.333781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.361070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1029/2079] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:28 2022] PUT /identity/v3/projects/44eac1bf35034b85b7404b4c3fd71140/users/ba51ce92ff494accbb420e1f03f143a0/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:51:28.364598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.365269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.365649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.402061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] MFA Rules not processed for user `ba51ce92ff494accbb420e1f03f143a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:28.403560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:28.484962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:28.543551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.546732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1051/2080] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:28.581888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:28.645673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d04917-545e-487d-acb2-c01e5d357aaa admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.659344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.660325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.660933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.661312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.662146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.664179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.699258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1030/2081] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:28 2022] POST /identity/v3/projects => generated 399 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:28.704107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.719038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50bad-30fa-4d86-b67f-ac6dd465d472 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.798400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.799070 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.799300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.799575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.800287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.802478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.837175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1052/2082] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:28 2022] POST /identity/v3/users => generated 306 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:28.841687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.857668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf667676-d92f-4f2a-ad30-c28f4113a85d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.866810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.867559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.867947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.868313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.869323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.871243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:28.887612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1031/2083] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:28.888480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:28.900525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c24f8-a34e-410a-aeb3-f3255f1f7217 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:28.939259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:28.941373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:28.943195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:28.943699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] PATH_INFO: `/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2/users/367b2b00413b40a281b15c6d1db65712/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:28.989300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd3c8ebcc674368abbf8a5f67752be2, user_id=367b2b00413b40a281b15c6d1db65712, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:28.996652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:29.054534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1053/2084] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:28 2022] PUT /identity/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2/users/367b2b00413b40a281b15c6d1db65712/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:29.058482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:29.076064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc9a86-9787-4b8b-90eb-63cb9ce196cc admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:29.161692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:29.161692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:29.161692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:29.161692 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:29.162248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:29.162248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684766d-933c-4ed7-b6a6-e0c027447ce2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:29.178933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1032/2085] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:29.181417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:29.198772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7c5a7-82f7-489c-beba-e249b7dc2cb3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:29.205166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:29.206491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:29.206792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:29.207073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] PATH_INFO: `/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2/users/367b2b00413b40a281b15c6d1db65712/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:29.209125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd3c8ebcc674368abbf8a5f67752be2, user_id=367b2b00413b40a281b15c6d1db65712, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:29.212210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:29.224502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1054/2086] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:29 2022] PUT /identity/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2/users/367b2b00413b40a281b15c6d1db65712/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:29.235187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:29.235431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:29.235711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:29.267042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] MFA Rules not processed for user `367b2b00413b40a281b15c6d1db65712`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:29.268335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:29.345685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1033/2087] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:29.356679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:29.358529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:29.420294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47db52-dd05-4b5e-ab34-2d9b49ebd220 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:29.425418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:29.426275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:29.426699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:29.427023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:29.437473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:29.438941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:29.448344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1055/2088] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:29.524689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:29.535317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ad7e-89ae-49e9-8475-0aed5b96c439 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:29.539269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:29.539883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:29.540160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:29.540433 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:29.555216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:29.603038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:29.604489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099a49-eb13-4d8a-b7eb-c7433d5394de service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:29.614388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1034/2089] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:37.099314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:37.138193 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:37.168007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b71f39-591d-437a-beb6-c1c9c65c6914 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:37.174266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:37.174938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:37.175223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:37.175498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] PATH_INFO: `/v3/users/ba51ce92ff494accbb420e1f03f143a0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:37.176676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba51ce92ff494accbb420e1f03f143a0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:37.178735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:37.237336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba51ce92ff494accbb420e1f03f143a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:37.238071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba51ce92ff494accbb420e1f03f143a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:37.246245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba51ce92ff494accbb420e1f03f143a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:37.261000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba51ce92ff494accbb420e1f03f143a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:37.271611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba51ce92ff494accbb420e1f03f143a0'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:37.280921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1056/2090] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:37 2022] DELETE /identity/v3/users/ba51ce92ff494accbb420e1f03f143a0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:37.285243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:37.285731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:37.285844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:37.306856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:37.308367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:37.376467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1035/2091] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:37.386994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:37.388555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:37.458125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1d92-4abc-4eab-a06c-153946791041 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:37.461811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:37.462487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:37.462773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:37.463053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:37.474075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:37.475483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:37.486919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1057/2092] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:37.938956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:37.940454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:37.953035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58f4e-6776-489f-9581-4d217c70dc12 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:37.957890 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:37.958473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:37.958757 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:37.959034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] PATH_INFO: `/v3/projects/44eac1bf35034b85b7404b4c3fd71140` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:37.960225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44eac1bf35034b85b7404b4c3fd71140)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:37.962165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:38.024054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44eac1bf35034b85b7404b4c3fd71140', 'request_id': 'req-c6c38ca6-ade3-4403-a1e5-640757025806'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.032359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44eac1bf35034b85b7404b4c3fd71140', 'request_id': 'req-c6c38ca6-ade3-4403-a1e5-640757025806'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.047049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44eac1bf35034b85b7404b4c3fd71140', 'request_id': 'req-c6c38ca6-ade3-4403-a1e5-640757025806'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.060607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] The token cache is being invalidate because project 44eac1bf35034b85b7404b4c3fd71140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:38.061030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.061905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.066471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1036/2093] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:37 2022] DELETE /identity/v3/projects/44eac1bf35034b85b7404b4c3fd71140 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:38.067667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:38.083606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bd9a4-65d8-4236-996b-195ab09c26de service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:38.153124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:38.168743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:38.173933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:38.173933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:38.173933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] PATH_INFO: `/v3/users/367b2b00413b40a281b15c6d1db65712` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:38.181242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367b2b00413b40a281b15c6d1db65712)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:38.186629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:38.291119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367b2b00413b40a281b15c6d1db65712'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.292352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367b2b00413b40a281b15c6d1db65712'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.302438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367b2b00413b40a281b15c6d1db65712'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.316379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367b2b00413b40a281b15c6d1db65712'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.330217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f1b665b7-fa19-4e02-97e7-1ca7341396b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367b2b00413b40a281b15c6d1db65712'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.338288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1058/2094] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:38 2022] DELETE /identity/v3/users/367b2b00413b40a281b15c6d1db65712 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:38.652883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:38.655729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:38.683516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38ca6-ade3-4403-a1e5-640757025806 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:38.746887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:38.767420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:38.768582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:38.768582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:38.768893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] PATH_INFO: `/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:38.772349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd3c8ebcc674368abbf8a5f67752be2)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:38.775369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:38.829247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd3c8ebcc674368abbf8a5f67752be2', 'request_id': 'req-e590a3fe-cafe-4db7-8506-5937119aaf94'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.835438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd3c8ebcc674368abbf8a5f67752be2', 'request_id': 'req-e590a3fe-cafe-4db7-8506-5937119aaf94'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.843147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd3c8ebcc674368abbf8a5f67752be2', 'request_id': 'req-e590a3fe-cafe-4db7-8506-5937119aaf94'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.853921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] The token cache is being invalidate because project 6bd3c8ebcc674368abbf8a5f67752be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:38.854206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.854909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e590a3fe-cafe-4db7-8506-5937119aaf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:38.857225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1037/2095] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:38 2022] DELETE /identity/v3/projects/6bd3c8ebcc674368abbf8a5f67752be2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:38.862039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:38.862389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:38.862595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:38.882813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:38.884062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:38.979068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1059/2096] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:38.980824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:38.981117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:38.982069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.004632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:39.005928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:39.018529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1038/2097] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:39.020252 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.031762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7df04-e1b2-4ec5-baa1-6a0f74551198 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.036637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.037438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.037815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.038214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.039030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:39.039465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.041385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.059097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1060/2098] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:39.060810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.072250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acd27d-ef07-40a9-ad47-447dd60a96ea None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.088039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.088581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.088862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.089134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.089837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.092282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.138380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1039/2099] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:39 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:39.139983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.151654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e8890-23e8-4f91-b19a-8d8ece79ac46 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.218275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.218928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.219296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.219648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.220689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.222734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.232575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:39.252146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1061/2100] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:39 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:39.253688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.272802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139ebff-b7a0-4e0b-afe6-c2afba2d7639 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.278402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.281871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.281871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.281871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.281871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.282944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.293596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1040/2101] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:39.300335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.314629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5514f-8006-4d52-87c2-0f67dd1a1801 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.324990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.325784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.326121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.326395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.327128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.328695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64547ce-e80d-4b4b-a117-5cca67e32f2a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.340306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1062/2102] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:39.346315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.357398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971357af-af7f-4145-9b4b-c65730ee52c9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.362465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.363163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.363445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.363722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] PATH_INFO: `/v3/projects/457215512d99444bb3dfbd78400c6700/users/55e798d688e2456c85455935f0b4d035/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.381485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457215512d99444bb3dfbd78400c6700, user_id=55e798d688e2456c85455935f0b4d035, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.384311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.400715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1041/2103] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:39 2022] PUT /identity/v3/projects/457215512d99444bb3dfbd78400c6700/users/55e798d688e2456c85455935f0b4d035/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:39.411731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.412026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.412301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.447395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] MFA Rules not processed for user `55e798d688e2456c85455935f0b4d035`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:39.448638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:39.526648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1063/2104] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:39.539487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.542544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:39.621114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5c76-8e38-482b-b0be-1a0c0e879327 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.625267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.625912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.626194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.626466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.635508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.636955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebee2-69a8-4a3f-b0b7-3456879a19f6 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.648030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1042/2105] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:39.669505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.736387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de254bf-5685-4d68-8933-167e30a86399 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:39.742106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:39.742750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.743032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.743314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] PATH_INFO: `/v3/users/55e798d688e2456c85455935f0b4d035` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.744504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e798d688e2456c85455935f0b4d035)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:39.746492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:39.798830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e798d688e2456c85455935f0b4d035'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:39.799572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e798d688e2456c85455935f0b4d035'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:39.811718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e798d688e2456c85455935f0b4d035'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:39.824330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e798d688e2456c85455935f0b4d035'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:39.826092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:39.835156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e798d688e2456c85455935f0b4d035'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:39.843839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1064/2106] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:39 2022] DELETE /identity/v3/users/55e798d688e2456c85455935f0b4d035 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:39.848710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:39.849138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:39.849230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:39.869987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:39.871199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:39.875482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:39.954110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1043/2107] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:39.963479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:39.969273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:40.049519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fde113-b011-4497-8ab6-57709821950b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:40.057364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:40.058040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:40.058353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:40.058732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:40.082098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:40.083573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca888-fcd3-4075-aa73-38108a45dc70 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:40.095185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1065/2108] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:40.616310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:40.617913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:40.648759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feded6c-465f-462c-9fb1-1cd957439a3d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:40.670730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:40.673453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:40.673453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:40.673808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] PATH_INFO: `/v3/projects/457215512d99444bb3dfbd78400c6700` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:40.675546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=457215512d99444bb3dfbd78400c6700)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:40.681285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:40.784251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457215512d99444bb3dfbd78400c6700', 'request_id': 'req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:40.793294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457215512d99444bb3dfbd78400c6700', 'request_id': 'req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:40.805355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457215512d99444bb3dfbd78400c6700', 'request_id': 'req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:40.819468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] The token cache is being invalidate because project 457215512d99444bb3dfbd78400c6700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:40.820011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:40.820532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bbd2c13d-a6a6-4a5a-911b-d7bb6aa58fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:40.822859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1044/2109] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:40 2022] DELETE /identity/v3/projects/457215512d99444bb3dfbd78400c6700 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:40.829342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:40.829953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:40.829953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:40.850760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:40.851973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:40.923441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1066/2110] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:40.931507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:40.931948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:40.932086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:40.958707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:40.961937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:40.980501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1045/2111] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:40.982339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:40.992569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149909ea-6f38-494d-8cde-b6b708e650a4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:40.998498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:40.999069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:40.999356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:40.999638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.000721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:41.002023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.004143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.020385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1067/2112] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:41.022108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.039748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243eeb6-f5b8-4ce6-b70d-e981285e4fda None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.045273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.045947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.046379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.046664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.047364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.049318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.079786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1046/2113] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:41.084108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.094211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79048-5334-4aa0-9ad7-4a5afcf41d86 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.161197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.161848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.162123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.162396 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.163065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.164970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.186756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1068/2114] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:41 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:41.190963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.204369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9c98a-e16d-4540-a7b1-beb1a41b57e0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.210282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.210858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.211141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.211412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.212116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.213720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.228937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1047/2115] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:41.232830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.243420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1911-2e86-41a6-ad57-2b44e48d0ed1 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.248278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.248918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.249305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.249611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.250412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.252078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1aaf9-f6f7-45fa-9613-9f4eaca06bec admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.254910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:41.268988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1069/2116] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:41.272942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.284215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0060-b832-42bc-b216-fce6ad6a3b8f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.289100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.289766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.290084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.290404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] PATH_INFO: `/v3/projects/6804124713e1452695af1a12a1e52b64/users/4e4fe6d4916a4acfaea7a17c79d32ca5/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.305322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6804124713e1452695af1a12a1e52b64, user_id=4e4fe6d4916a4acfaea7a17c79d32ca5, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.308064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741c975-f224-4bfa-896c-5410f77eaf27 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.323227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1048/2117] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:41 2022] PUT /identity/v3/projects/6804124713e1452695af1a12a1e52b64/users/4e4fe6d4916a4acfaea7a17c79d32ca5/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:41.335065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.335354 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.335636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.358717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5f51-ebb0-48da-a621-a97311ea9baa None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.359061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5f51-ebb0-48da-a621-a97311ea9baa None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.359286 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5f51-ebb0-48da-a621-a97311ea9baa None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.360895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1049/2118] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:51:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:51:41.386095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] MFA Rules not processed for user `4e4fe6d4916a4acfaea7a17c79d32ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:41.387361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:41.392780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.393248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.394110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.420024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:41.421481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:41.456426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:41.498231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1070/2119] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:41.498231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.498231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:41.530906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1050/2120] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:51:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:41.531843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.533398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:41.565386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f6e92-bada-4020-8f8d-778ac73789b3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.570113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.570758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.571043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.571323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] PATH_INFO: `/v3/users/b3a4fdcc0dde49009f4e55d25af4da7c` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.572506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a4fdcc0dde49009f4e55d25af4da7c)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.574478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.590403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2fa6-cf4e-48a9-a806-0ed4178c81d2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.594685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.595303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.595866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.596121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.605459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.610826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.618484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1051/2121] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:41.619508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.621043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:41.633291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a4fdcc0dde49009f4e55d25af4da7c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:41.634216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a4fdcc0dde49009f4e55d25af4da7c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:41.642716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6bd47-e8c7-4a56-8dd1-18b18b41a84b service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:41.643757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a4fdcc0dde49009f4e55d25af4da7c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:41.648346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:41.649068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:41.649419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:41.649766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:41.657949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a4fdcc0dde49009f4e55d25af4da7c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:41.658824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:41.660483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:41.665478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-5793b92e-a2fa-41a9-8c1b-628c62d98a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a4fdcc0dde49009f4e55d25af4da7c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:41.671502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1052/2122] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:41.673020 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1071/2123] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:41 2022] DELETE /identity/v3/users/b3a4fdcc0dde49009f4e55d25af4da7c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:41.879060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:41.922855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7c3c-0e93-4aa4-bd10-4c9b95655c10 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.038790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.039487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.039774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.040053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] PATH_INFO: `/v3/projects/f9210aacd9bd4593a85429fe1768e28d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.041248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9210aacd9bd4593a85429fe1768e28d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.054203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.107382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9210aacd9bd4593a85429fe1768e28d', 'request_id': 'req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:42.114844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9210aacd9bd4593a85429fe1768e28d', 'request_id': 'req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:42.123684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:42.129346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9210aacd9bd4593a85429fe1768e28d', 'request_id': 'req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:42.141063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] The token cache is being invalidate because project f9210aacd9bd4593a85429fe1768e28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:42.141358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:42.142015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3366a21d-8f7b-4dbb-84d4-2a21b5ee89a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:42.143612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1053/2124] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:41 2022] DELETE /identity/v3/projects/f9210aacd9bd4593a85429fe1768e28d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:42.148708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.149178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.149275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.163736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:42.171084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:42.172928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:42.237992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1072/2125] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:42.246535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.246832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.247673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.267568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:42.268822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:42.278095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1054/2126] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:42.282092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.291654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a4658-71c8-4f04-9501-25a21ed1834b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.296767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.297377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.297660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.297962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.298658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:42.298965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.300658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.312732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1073/2127] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:42.317144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.326309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059f966-f25c-4da0-9b57-9855d760b3d0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.330877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.331498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.331776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.332045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.332697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.334729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.364844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1055/2128] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:42 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:42.369188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.379753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4c9e-069b-414e-ad39-9cdbb86404bc admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.440028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.440670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.440960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.442091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.442091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.443856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.469082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1074/2129] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:42 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:42.470601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.482601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c7cc-b610-408a-b8a3-c8847b907728 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.487241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.487849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.488132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.488405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.489094 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.490704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.501944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1056/2130] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:42.506602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.517465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c255-d492-4a30-b587-f49f361bc850 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.522476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.523176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.523533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.525646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.525646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.526928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211a93-5317-4c7a-9513-aa0b1083dbb9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.542126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1075/2131] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:42.543838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.554423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a2c9d-a55c-4c45-85c5-48bb2b670cfb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.574015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.574901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.575416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.575956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] PATH_INFO: `/v3/projects/f36dffd2ab1643ed939ffe87e5928187/users/03f312d5c7334ccc82be17c5dfe29aeb/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.594696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36dffd2ab1643ed939ffe87e5928187, user_id=03f312d5c7334ccc82be17c5dfe29aeb, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.597809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.614430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1057/2132] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:42 2022] PUT /identity/v3/projects/f36dffd2ab1643ed939ffe87e5928187/users/03f312d5c7334ccc82be17c5dfe29aeb/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:42.619133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.619423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.619751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.640806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:42.642062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:42.734473 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1076/2133] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:42.740345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:42.742778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:42.813315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc1d-72e3-4100-a9e3-0e701b8fac3d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:42.817584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:42.818325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:42.818614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:42.818893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:42.830243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:42.831773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571213b-e391-4c23-a3d3-ab25d5c15543 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:42.842071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1058/2134] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:46.624810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:46.638609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:46.662128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ad93c-8d73-4d17-967f-fdb0bcc24448 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:46.670649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:46.671380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:46.671666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:46.671947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] PATH_INFO: `/v3/users/d4265c18590c41ec9e8c43053104da12` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:46.673346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4265c18590c41ec9e8c43053104da12)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:46.675366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:46.696054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:46.782363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4265c18590c41ec9e8c43053104da12'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:46.785143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4265c18590c41ec9e8c43053104da12'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:46.798729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4265c18590c41ec9e8c43053104da12'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:46.818283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4265c18590c41ec9e8c43053104da12'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:46.830403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4265c18590c41ec9e8c43053104da12'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:46.841940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1077/2135] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:46 2022] DELETE /identity/v3/users/d4265c18590c41ec9e8c43053104da12 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:46.843981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:46.844461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:46.844720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:46.880594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:46.881943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:47.005497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1059/2136] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:47.016905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:47.018484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:47.141601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69590277-272a-4ec6-be3b-c9f1e92a1e7c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:47.154284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:47.155621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:47.155621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:47.156780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:47.178774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:47.180424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695f209-a1a1-40ed-9234-7a1a0bcdbe76 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:47.221595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1078/2137] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:47.683129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:47.690167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:47.728556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738bbfe-a188-4b4a-aa5c-353b1d35194b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:47.752369 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:47.753294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:47.753731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:47.761359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] PATH_INFO: `/v3/projects/db49de5691974be28efa13dd4e0d2d2b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:47.763374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db49de5691974be28efa13dd4e0d2d2b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:47.765655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:47.802671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:47.851958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db49de5691974be28efa13dd4e0d2d2b', 'request_id': 'req-81a045b5-a945-4ea2-a1bf-689a64a54cb1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:47.859909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db49de5691974be28efa13dd4e0d2d2b', 'request_id': 'req-81a045b5-a945-4ea2-a1bf-689a64a54cb1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:47.875849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db49de5691974be28efa13dd4e0d2d2b', 'request_id': 'req-81a045b5-a945-4ea2-a1bf-689a64a54cb1'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:47.897393 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] The token cache is being invalidate because project db49de5691974be28efa13dd4e0d2d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:47.898090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:47.901343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81a045b5-a945-4ea2-a1bf-689a64a54cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:47.911589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1060/2138] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:47 2022] DELETE /identity/v3/projects/db49de5691974be28efa13dd4e0d2d2b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:47.916657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3395f61-b9fb-4b69-8c24-66bbe19f8f99 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:47.917034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3395f61-b9fb-4b69-8c24-66bbe19f8f99 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:47.917518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3395f61-b9fb-4b69-8c24-66bbe19f8f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:47.942104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c3395f61-b9fb-4b69-8c24-66bbe19f8f99 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:47.943329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c3395f61-b9fb-4b69-8c24-66bbe19f8f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:47.973978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:47.974277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:47.974551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.018910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] MFA Rules not processed for user `03f312d5c7334ccc82be17c5dfe29aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:48.020788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:48.028686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1079/2139] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:48.039041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.039423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.040408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.068629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:48.069892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:48.082745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1080/2140] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:48.084332 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.096486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a228d9-093f-416e-abac-e04a9ad2fbc0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.110561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.110561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.110561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.110561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.110917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:51:48.111007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1061/2141] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:48.111103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.112087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.129986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1081/2142] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:51:48.134012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.139451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.141201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:48.147914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8636a7-43f0-4f9a-ba50-6f362b5ef665 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.154104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:48.154804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.155675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.156145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.156493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.157304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.160379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.213397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1062/2143] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:48.217204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.276261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a03f0c-7657-4021-8c74-24b412fab96f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.319760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf034455-f945-4fe4-ac29-59dd6fc3bde6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.325717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.326491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.326877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.327252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.328024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.330287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.359031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1063/2144] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:48.363119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.364905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.365943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.366271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.366618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.377149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:48.390419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62b196-9ff8-461e-a5ba-8ed6ce0c93fa admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.396974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.398151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.398598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.398932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.399775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.401607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.414120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1064/2145] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:48.417906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.428086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e2624-4ac6-4eee-b051-16ef88c232eb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.432910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.433515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.433800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.434786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.435528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.437564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.453578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1065/2146] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:48.455166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.471376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.473278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd5857-fe0e-4f21-ac4d-4e944ef56095 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.474072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f557-2fd8-4b64-9490-18886be9b40c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.481462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.487842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.502950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.503248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] PATH_INFO: `/v3/projects/cd2e41f0a0c2480eb8477bb201d2b66f/users/32f57b47d8ce424f8b034df53352b8d5/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.503509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1082/2147] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:48.521415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2e41f0a0c2480eb8477bb201d2b66f, user_id=32f57b47d8ce424f8b034df53352b8d5, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.524206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.541288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1066/2148] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:48 2022] PUT /identity/v3/projects/cd2e41f0a0c2480eb8477bb201d2b66f/users/32f57b47d8ce424f8b034df53352b8d5/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:48.566696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.566696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.566696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.600026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] MFA Rules not processed for user `32f57b47d8ce424f8b034df53352b8d5`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:48.601290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:48.663300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:48.708648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1083/2149] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:48.712830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.806379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442433aa-287e-497e-a15b-d24d72c95994 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:48.812175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:48.813187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:48.813327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:48.813926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:48.814809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:48.817289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:48.901800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1067/2150] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:48.908183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:48.924091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759db16-3a83-41b0-af5d-eea2f75bf63d None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.035100 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.035808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.036333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.036619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.037786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.044551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.095257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1084/2151] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:48 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:49.099186 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.126512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73fa4-c678-4f55-897a-47c6fca5e396 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.136063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.136859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.137811 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.139108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.139914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.142807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.157805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1068/2152] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:49.161769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.173511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae235b-6c32-412e-bfd0-e01e52ff729f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.178442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.179157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.179512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.179860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.180647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.182370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0358f-694c-4e96-9d4a-f1a47c44bdd0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.219211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1085/2153] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:49.220711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.234978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5c329-7d92-49fe-9770-3ecd9befa106 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.247907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.249100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.249530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.249749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] PATH_INFO: `/v3/projects/4b351485b0804a1b9abef058e5093c5f/users/fc2e762767d744349f49a91dc5e03d63/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.269453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b351485b0804a1b9abef058e5093c5f, user_id=fc2e762767d744349f49a91dc5e03d63, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.273258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.308220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1069/2154] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:49 2022] PUT /identity/v3/projects/4b351485b0804a1b9abef058e5093c5f/users/fc2e762767d744349f49a91dc5e03d63/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:49.319038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.319325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.319608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.362857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] MFA Rules not processed for user `fc2e762767d744349f49a91dc5e03d63`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:49.364099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:49.515281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1086/2155] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:49.519963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.621173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e18f-08ce-4da0-8496-55c0e84a289d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.627898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.628529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.628817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.629095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.629772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.631827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.647931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:49.664617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1070/2156] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:51:49 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:51:49.669122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.684531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea12a-d441-4caa-94cf-bdf6436ecd66 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.737154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:49.755519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.756147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.756431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.756836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.757684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.759992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.785597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1087/2157] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:51:49 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:51:49.786826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.803007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eda17-3174-4cc8-a31d-1b36a8b3fde8 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.810245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.810881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.811165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.811441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.812149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.813975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.827244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1071/2158] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:51:49.831363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.843657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a499-1187-4340-a4de-7de301654e3c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.850140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.851161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.851161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.851357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] PATH_INFO: `/v3/projects/bd5b5be4276340e488ab2d694b942249/users/76bb30629f8b462d9bc6ae1d382d6b52/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.871182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5b5be4276340e488ab2d694b942249, user_id=76bb30629f8b462d9bc6ae1d382d6b52, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.874382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.887754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1088/2159] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:49 2022] PUT /identity/v3/projects/bd5b5be4276340e488ab2d694b942249/users/76bb30629f8b462d9bc6ae1d382d6b52/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:49.892035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:49.904896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc2341-7835-45d0-8f89-81d8f9848bf6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:49.977058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:49.977703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:49.978012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:49.978291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:49.979141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:49.981198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f8db-8d35-4eb0-8e8e-86901b8220ad admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:49.993856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1072/2160] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:51:49.997643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:50.009802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f70495-9c2e-4ad8-bc8e-420c266f5349 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:50.015114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:50.015813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:50.016101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:50.016387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] PATH_INFO: `/v3/projects/bd5b5be4276340e488ab2d694b942249/users/76bb30629f8b462d9bc6ae1d382d6b52/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:50.019512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5b5be4276340e488ab2d694b942249, user_id=76bb30629f8b462d9bc6ae1d382d6b52, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:50.022795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee183-af60-473b-821f-483f50c834f1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:50.039181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1089/2161] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:51:49 2022] PUT /identity/v3/projects/bd5b5be4276340e488ab2d694b942249/users/76bb30629f8b462d9bc6ae1d382d6b52/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:50.049578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:50.049966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:50.050272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:50.081055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] MFA Rules not processed for user `76bb30629f8b462d9bc6ae1d382d6b52`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:50.082540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:50.163218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:50.163506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:50.163788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:50.169923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1073/2162] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:50.181467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:50.182916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:50.184963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] MFA Rules not processed for user `03f312d5c7334ccc82be17c5dfe29aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:50.186750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:50.268912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1090/2163] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:51:50.283865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240b2ee-4fb9-42b2-839c-0c8977bb90d3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:50.289358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:50.290102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:50.290323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:50.290589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:50.333267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:50.439668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:50.446861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:50.472172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1074/2164] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:52.185311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:52.186833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:52.211605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e563-b230-47f6-8963-b8cbba59e6f2 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:52.215677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:52.216313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:52.216600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:52.216878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:52.330294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:52.332542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:52.350039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1091/2165] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:53.171663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:53.173242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:53.190030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:53.201399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b839b-6868-4284-ad38-a2faeca8af0b service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:53.207226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:53.208309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:53.208593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:53.208869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:53.221718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:53.224682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1eeda-0ae9-4498-9754-07f7b98fb06d service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:53.235878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1092/2166] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:51:53.258835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd42220-0cc5-48ff-833f-39370dc024d2 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:53.266504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:53.267264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:53.268121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:53.268399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] PATH_INFO: `/v3/users/4e4fe6d4916a4acfaea7a17c79d32ca5` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:53.270296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4fe6d4916a4acfaea7a17c79d32ca5)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:53.272331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:53.335425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4fe6d4916a4acfaea7a17c79d32ca5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:53.345800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4fe6d4916a4acfaea7a17c79d32ca5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:53.346554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4fe6d4916a4acfaea7a17c79d32ca5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:53.368650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4fe6d4916a4acfaea7a17c79d32ca5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:53.369650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4fe6d4916a4acfaea7a17c79d32ca5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:53.390174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1075/2167] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:51:53 2022] DELETE /identity/v3/users/4e4fe6d4916a4acfaea7a17c79d32ca5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:51:53.395198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:53.395944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:53.396278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:53.420829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:51:53.422248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:53.447989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:53.506488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1093/2168] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:51:53.516756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:53.518868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:53.578338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:51:53.603683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa480a3f-d2e0-48eb-98b7-d41536a021ab admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:53.609343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:53.610906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:53.614115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:53.614115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:53.631481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:53.634320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:53.664315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1076/2169] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:51:54.164357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:51:54.166224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:51:54.180452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5906bc-c633-40a2-a388-789845f98220 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:51:54.185717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:51:54.186761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:51:54.187229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:51:54.187680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] PATH_INFO: `/v3/projects/6804124713e1452695af1a12a1e52b64` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:51:54.189038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6804124713e1452695af1a12a1e52b64)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:51:54.191205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:51:54.269379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6804124713e1452695af1a12a1e52b64', 'request_id': 'req-c723bcab-08a8-41dc-8052-6dcbdeb28d54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:54.276690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6804124713e1452695af1a12a1e52b64', 'request_id': 'req-c723bcab-08a8-41dc-8052-6dcbdeb28d54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:54.286171 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6804124713e1452695af1a12a1e52b64', 'request_id': 'req-c723bcab-08a8-41dc-8052-6dcbdeb28d54'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:54.300753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] The token cache is being invalidate because project 6804124713e1452695af1a12a1e52b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:51:54.301291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:54.302347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c723bcab-08a8-41dc-8052-6dcbdeb28d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:51:54.304181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1094/2170] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:51:54 2022] DELETE /identity/v3/projects/6804124713e1452695af1a12a1e52b64 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:02.623789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:02.625496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:02.645563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d54-4070-4af0-980d-37664e97e140 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:02.719652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:02.720776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:02.721478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:02.721905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] PATH_INFO: `/v3/users/32f57b47d8ce424f8b034df53352b8d5` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:02.723355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f57b47d8ce424f8b034df53352b8d5)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:02.725447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:02.778550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f57b47d8ce424f8b034df53352b8d5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:02.786138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f57b47d8ce424f8b034df53352b8d5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:02.787205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f57b47d8ce424f8b034df53352b8d5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:02.801847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f57b47d8ce424f8b034df53352b8d5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:02.802583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f57b47d8ce424f8b034df53352b8d5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:02.821932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1077/2171] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:02 2022] DELETE /identity/v3/users/32f57b47d8ce424f8b034df53352b8d5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:02.824105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:02.824697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:02.824794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:02.847488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:02.848716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:02.925604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1095/2172] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:02.932082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:02.933585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:03.002053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bf92-8c49-4d9b-aec2-248052568376 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:03.007434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:03.008524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:03.009040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:03.009497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:03.025145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:03.026763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:03.039750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1078/2173] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:52:03.555582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:03.557113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:03.578587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5997a8-1d6c-4473-baf5-26ebb7e38932 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:03.595163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:03.596121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:03.596406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:03.596682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] PATH_INFO: `/v3/projects/cd2e41f0a0c2480eb8477bb201d2b66f` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:03.598360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2e41f0a0c2480eb8477bb201d2b66f)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:03.600597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:03.704844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:03.711211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2e41f0a0c2480eb8477bb201d2b66f', 'request_id': 'req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.724286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2e41f0a0c2480eb8477bb201d2b66f', 'request_id': 'req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.735684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2e41f0a0c2480eb8477bb201d2b66f', 'request_id': 'req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.752135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] The token cache is being invalidate because project cd2e41f0a0c2480eb8477bb201d2b66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:03.753075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.753190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.758073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1096/2174] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:03 2022] DELETE /identity/v3/projects/cd2e41f0a0c2480eb8477bb201d2b66f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:03.759899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:03.766161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:03.780799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20671-4075-4f22-963a-8903ae5db0a2 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:03.857208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:03.858233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:03.858522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:03.858798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] PATH_INFO: `/v3/users/fc2e762767d744349f49a91dc5e03d63` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:03.860254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2e762767d744349f49a91dc5e03d63)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:03.865455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:03.920969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2e762767d744349f49a91dc5e03d63'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.929118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2e762767d744349f49a91dc5e03d63'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.929950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2e762767d744349f49a91dc5e03d63'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.944564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2e762767d744349f49a91dc5e03d63'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.945314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2e762767d744349f49a91dc5e03d63'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:03.960933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1079/2175] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:03 2022] DELETE /identity/v3/users/fc2e762767d744349f49a91dc5e03d63 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:04.263732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:04.265661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:04.288639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781ec3-fa41-4bc0-830e-fc24d7fb0be3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:04.353569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:04.354325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:04.354702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:04.355054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] PATH_INFO: `/v3/projects/4b351485b0804a1b9abef058e5093c5f` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:04.356861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b351485b0804a1b9abef058e5093c5f)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:04.358881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:04.434842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b351485b0804a1b9abef058e5093c5f', 'request_id': 'req-43de165f-cac1-441e-9849-f03367474a3a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.444211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b351485b0804a1b9abef058e5093c5f', 'request_id': 'req-43de165f-cac1-441e-9849-f03367474a3a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.461808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b351485b0804a1b9abef058e5093c5f', 'request_id': 'req-43de165f-cac1-441e-9849-f03367474a3a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.482319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] The token cache is being invalidate because project 4b351485b0804a1b9abef058e5093c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:04.482856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.483711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.485699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1097/2176] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:04 2022] DELETE /identity/v3/projects/4b351485b0804a1b9abef058e5093c5f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:04.489998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:04.492236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:04.506920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5492b-5a64-4448-940e-9aca61d3d119 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:04.578557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:04.579420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:04.579778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:04.580133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] PATH_INFO: `/v3/users/76bb30629f8b462d9bc6ae1d382d6b52` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:04.581529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bb30629f8b462d9bc6ae1d382d6b52)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:04.583551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:04.612344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:04.648730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bb30629f8b462d9bc6ae1d382d6b52'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.659781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bb30629f8b462d9bc6ae1d382d6b52'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.660858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bb30629f8b462d9bc6ae1d382d6b52'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.691293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bb30629f8b462d9bc6ae1d382d6b52'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.692418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3569d16f-20fb-4631-898b-86fee0245fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bb30629f8b462d9bc6ae1d382d6b52'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:04.720001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1080/2177] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:04 2022] DELETE /identity/v3/users/76bb30629f8b462d9bc6ae1d382d6b52 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:05.098367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.099911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:05.117071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de165f-cac1-441e-9849-f03367474a3a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.189335 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.190019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.190304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.190583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] PATH_INFO: `/v3/projects/bd5b5be4276340e488ab2d694b942249` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.191734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5b5be4276340e488ab2d694b942249)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.193636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.195907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:05.243372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5b5be4276340e488ab2d694b942249', 'request_id': 'req-d1b44a44-9148-4f93-9e48-9b55658774de'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:05.248860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5b5be4276340e488ab2d694b942249', 'request_id': 'req-d1b44a44-9148-4f93-9e48-9b55658774de'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:05.256287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5b5be4276340e488ab2d694b942249', 'request_id': 'req-d1b44a44-9148-4f93-9e48-9b55658774de'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:05.268074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] The token cache is being invalidate because project bd5b5be4276340e488ab2d694b942249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:05.268620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:05.269131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1b44a44-9148-4f93-9e48-9b55658774de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:05.270819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1098/2178] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:05 2022] DELETE /identity/v3/projects/bd5b5be4276340e488ab2d694b942249 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:05.275876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.276293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.276504 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.296478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:05.297663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:05.356889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1081/2179] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:05.424326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.424613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.425089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.445185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:05.446427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:05.454820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1099/2180] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:05.458993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.469639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01fce2-211f-4184-ab68-59d90e582c4e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.474552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.475155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.475441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.475716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.476417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:52:05.476728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.478354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.490062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1082/2181] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:52:05.494035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.503142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee8e92-18f7-4c1c-b0de-bc97ad562488 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.507779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.508381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.508665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.508944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.509616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.511544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.540869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1100/2182] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:05.545085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.554987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87492ffd-ac98-41a4-bbbe-01d0fbdbe9d3 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.612513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.613512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.613512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.613704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.614383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.616377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.642747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1083/2183] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:52:05 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:05.644116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.654381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce643b52-1e8c-45f7-b9d8-cb12cea92463 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.659105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.659786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.660133 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.660475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.661263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.662972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.673674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1101/2184] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:05.677614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.688333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c9797-bf4d-4662-bd8f-e335ffc89096 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.693001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.693609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.693939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.694215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.694946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.696545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd982be-e33d-4986-8909-85be5395d40c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.707563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1084/2185] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:52:05.711556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.724847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538447e-5fb6-4aa7-838f-21cb81c622f1 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.730255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.730977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.731261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.731545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] PATH_INFO: `/v3/projects/15f411d81f95464389c9ea3c12cee88d/users/07220936c8784339b62f22482264e234/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.747338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f411d81f95464389c9ea3c12cee88d, user_id=07220936c8784339b62f22482264e234, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.750014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.766888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1102/2186] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:05 2022] PUT /identity/v3/projects/15f411d81f95464389c9ea3c12cee88d/users/07220936c8784339b62f22482264e234/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:52:05.772008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ee5c-3399-47b2-bad6-039be77fe102 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.772297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ee5c-3399-47b2-bad6-039be77fe102 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.772568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ee5c-3399-47b2-bad6-039be77fe102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.786923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:05.793057 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-2bf0ee5c-3399-47b2-bad6-039be77fe102 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:05.794972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2bf0ee5c-3399-47b2-bad6-039be77fe102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:05.861595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1085/2187] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:52:05.873768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:05.875620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:05.935480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34509e6-9517-4bf4-ac3f-47deb6be38c0 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:05.939191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:05.939812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:05.940121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:05.940446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:05.950264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:05.951813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:05.961263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1103/2188] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:52:11.991251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:11.991823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:11.991920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.023138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] MFA Rules not processed for user `07220936c8784339b62f22482264e234`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:12.024379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:12.108471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1086/2189] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:12.112943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.123018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a2fd-e49a-459a-b6a3-3348e5467a96 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.130049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.130761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.131111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.131461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.132204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.134228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.172305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.174042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1104/2190] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:52:12 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:12.184540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f5c70-1a30-4d3e-9ba5-debc531a8c1f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.283897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.284888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.285565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.286017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.287093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.289471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.321655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1087/2191] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:52:12 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:12.325360 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.338108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a14c-c877-4d83-a4df-977fe56659ad admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.351485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.352218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.352572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.352920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.353697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.355993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.381608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:12.405369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1105/2192] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:12.406714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.427017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951ac45-3d73-492f-afe3-866c7d424e1b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.443297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.445269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.445775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.446180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] PATH_INFO: `/v3/projects/804531b781f24296886ce74b67637d23/users/30243c3bccab4a739cfbe1ff208c1af3/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.492831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=804531b781f24296886ce74b67637d23, user_id=30243c3bccab4a739cfbe1ff208c1af3, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.496185 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.523263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1088/2193] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:12 2022] PUT /identity/v3/projects/804531b781f24296886ce74b67637d23/users/30243c3bccab4a739cfbe1ff208c1af3/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:12.527047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.545687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061754-74c6-4e7e-8cc2-e3eb3018d467 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.646432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.647085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.647374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.647658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.648527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.650192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.665011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1106/2194] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:12.669117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.679818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:12.687064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf12af-069b-4c5e-a2be-05a1ba042c0d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.693939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.694799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.695180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.695552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] PATH_INFO: `/v3/projects/804531b781f24296886ce74b67637d23/users/30243c3bccab4a739cfbe1ff208c1af3/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.701529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] RBAC: Authorizing `identity:create_grant(project_id=804531b781f24296886ce74b67637d23, user_id=30243c3bccab4a739cfbe1ff208c1af3, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.714572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b3e4e-a3c9-472a-a309-9cf1344386da admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.748237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:12.749593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1089/2195] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:12 2022] PUT /identity/v3/projects/804531b781f24296886ce74b67637d23/users/30243c3bccab4a739cfbe1ff208c1af3/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:12.753780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:12.777550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0112c-5c68-4f17-bbb9-c702f2944dca admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:12.888397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:12.890458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:12.890754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:12.892613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] PATH_INFO: `/v3/users/71af650c5ddd4622a2b8ae2b9bc385bf` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:12.894934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=71af650c5ddd4622a2b8ae2b9bc385bf)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:12.897103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:12.994570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71af650c5ddd4622a2b8ae2b9bc385bf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:12.995618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71af650c5ddd4622a2b8ae2b9bc385bf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:13.015158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71af650c5ddd4622a2b8ae2b9bc385bf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:13.046906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71af650c5ddd4622a2b8ae2b9bc385bf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:13.061726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71af650c5ddd4622a2b8ae2b9bc385bf'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:13.077411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1107/2196] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:12 2022] DELETE /identity/v3/users/71af650c5ddd4622a2b8ae2b9bc385bf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:52:13.079387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:13.079847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:13.080332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:13.105154 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:13.106925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:13.205734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1090/2197] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:52:13.216969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:13.219123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:13.244053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:13.315261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5722a-4ccd-4464-96f2-7f047db1f5de admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:13.320389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:13.321896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:13.322406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:13.322845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:13.336338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:13.338183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102384c0-bd56-4042-8220-eda432a91f20 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:13.354012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1108/2198] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:52:13.984324 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:13.985813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:14.027919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d93311-653c-4d8a-aa1c-1d5ecf7a885a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.035719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.036600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.036889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.037325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] PATH_INFO: `/v3/projects/c62ccb69bf5240f6a042708137e9bef6` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.038631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62ccb69bf5240f6a042708137e9bef6)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.040570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.142497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62ccb69bf5240f6a042708137e9bef6', 'request_id': 'req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:14.152329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62ccb69bf5240f6a042708137e9bef6', 'request_id': 'req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:14.176754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62ccb69bf5240f6a042708137e9bef6', 'request_id': 'req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:14.223590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] The token cache is being invalidate because project c62ccb69bf5240f6a042708137e9bef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:14.224184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:14.226804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3c2d95a1-1fcd-4a70-a171-480f2aadb2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:14.233931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1091/2199] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:13 2022] DELETE /identity/v3/projects/c62ccb69bf5240f6a042708137e9bef6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:14.238749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.239230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.239393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.285437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:14.299545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:14.432346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1109/2200] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:14.440021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.440340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.440629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.473172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:14.477743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:14.492594 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1092/2201] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:14.493150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.507225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2605-562f-4434-8266-d6adba4fa252 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.516765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.517893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.518334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.518842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.520006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:52:14.520689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.522717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.545656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.557057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1110/2202] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:52:14.561061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d48833-5c53-4b94-8631-9d386c183fe8 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.567715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.568715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.569146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.569559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.571159 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.573469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.617561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1093/2203] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:52:14 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:14.623643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.635649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1026-aca2-40d3-9e2a-959613a69272 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.707983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.707983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.708690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.709063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.709880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.721496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.754122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1111/2204] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:52:14 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:14.758137 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.778029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a415d0-b274-4964-b247-25807d5b3e98 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.789356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.790573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.791296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.791740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.793021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.796406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.818223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1094/2205] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:14.819880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.832422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c16756-ce6a-4e9f-8b2f-61287d55dcec admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.838579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.839345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.840231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.840578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.841355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.843783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b7ce8-f5a4-49bd-9c23-1e1b514064b7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.856385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1112/2206] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:52:14.860809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:14.871928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76972-12c2-4c50-809a-c9b5d5932ca5 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:14.878358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:14.879283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.879694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.880149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] PATH_INFO: `/v3/projects/fe15cad5decb438cb64d1742537690fb/users/3d960290efab417eb29ff05ad6c3144b/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.898116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe15cad5decb438cb64d1742537690fb, user_id=3d960290efab417eb29ff05ad6c3144b, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:14.900898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:14.917554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1095/2207] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:14 2022] PUT /identity/v3/projects/fe15cad5decb438cb64d1742537690fb/users/3d960290efab417eb29ff05ad6c3144b/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:14.930126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:14.930524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:14.930828 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:14.963876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] MFA Rules not processed for user `3d960290efab417eb29ff05ad6c3144b`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:14.965592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:14.977342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:15.052682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1113/2208] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:15.060561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:15.062462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:15.076131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:15.136073 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f2bb5-9b55-4626-a5c4-50debed5219d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:15.140576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:15.141397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:15.141709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:15.142034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:15.152427 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:15.154016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179922c6-7c4e-4be2-85a7-9c7400dba38c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:15.164723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1096/2209] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:52:18.078272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:18.180422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa462c-60da-41ee-a709-262ecbccecd8 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:18.186632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:18.187465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:18.187853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:18.188233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] PATH_INFO: `/v3/users/3d960290efab417eb29ff05ad6c3144b` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:18.189928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d960290efab417eb29ff05ad6c3144b)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:18.192684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:18.303905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d960290efab417eb29ff05ad6c3144b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:18.309867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d960290efab417eb29ff05ad6c3144b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:18.321513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d960290efab417eb29ff05ad6c3144b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:18.345946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d960290efab417eb29ff05ad6c3144b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:18.357784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b8b8609a-e257-49aa-b0c0-666972b8ddad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d960290efab417eb29ff05ad6c3144b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:18.369742 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1114/2210] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:18 2022] DELETE /identity/v3/users/3d960290efab417eb29ff05ad6c3144b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:18.372093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:18.373010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:18.373605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:18.399459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:18.400920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:18.461368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:18.462497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:18.462782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:18.504154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] MFA Rules not processed for user `30243c3bccab4a739cfbe1ff208c1af3`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:18.505441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:18.515974 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1097/2211] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:18.526076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:18.528340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:18.547361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:18.669765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c68e0-6740-4dea-bef1-d6bf565a649b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:18.671220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1115/2212] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:52:18.674954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:18.675872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:18.675872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:18.676124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:18.687668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:18.689105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:18.697293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:18.699121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:18.710697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1098/2213] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:52:18.802320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2e98-d4ec-45f7-85e7-3359524a4f99 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:18.808906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:18.809651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:18.810067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:18.810460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:18.812921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:18.888867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:18.890714 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54af30-351d-4c64-a1d8-e1e9b25b5df0 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:18.901207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1116/2214] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:52:19.061859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.062627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:19.078209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70982c9-5b0f-4793-8b51-19ed43a9837e service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.083508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.084215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.084530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.084835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] PATH_INFO: `/v3/projects/fe15cad5decb438cb64d1742537690fb` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.086138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe15cad5decb438cb64d1742537690fb)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.089127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.170134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe15cad5decb438cb64d1742537690fb', 'request_id': 'req-62c6d765-bd02-4a39-aeb3-f5d413219080'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:19.170134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:19.179682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe15cad5decb438cb64d1742537690fb', 'request_id': 'req-62c6d765-bd02-4a39-aeb3-f5d413219080'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:19.191251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe15cad5decb438cb64d1742537690fb', 'request_id': 'req-62c6d765-bd02-4a39-aeb3-f5d413219080'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:19.208795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] The token cache is being invalidate because project fe15cad5decb438cb64d1742537690fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:19.209142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:19.209919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-62c6d765-bd02-4a39-aeb3-f5d413219080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:19.217615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.217615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.217615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.221017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1099/2215] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:19 2022] DELETE /identity/v3/projects/fe15cad5decb438cb64d1742537690fb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:19.240158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:19.241394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:19.299163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:19.326758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1117/2216] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:19.328836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.329482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.329959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.357407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:19.358877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:19.367564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1100/2217] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:19.372290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.382399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dd6c3-6367-421f-85c6-1914c4249660 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.387783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.388505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.388851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.389197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.390011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:52:19.390551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.392420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.409029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1118/2218] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:52:19.410298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.421628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6704f1-b356-4d15-9874-912b59266410 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.426774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.427788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.428215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.428681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.429530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.431855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.471422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1101/2219] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:52:19 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:52:19.475443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.495629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962cfe-3a15-46c6-87ad-6883399874f7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.576720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.577639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.579531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.579804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.580581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.582735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.609352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1119/2220] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:52:19 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:52:19.614091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.634130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43a6c-f50c-47cd-bd95-0b39ba804372 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.641482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.642330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.642928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.643333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.646106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.648472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.663664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1102/2221] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:19.668617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.690826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5ef05-af5c-4fe7-a7f6-cb4d44a4f123 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.696922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.697747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.698268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.698624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.699855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.701487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a70df-33ea-489b-8072-1a30e0e6c9ef admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.736166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:19.747295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1120/2222] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:19.751386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfcf3f-6993-40e8-9478-257a9015cdce admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:19.758126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:19.758933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.759440 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.759814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] PATH_INFO: `/v3/projects/048547025e6f4cbaba52cbc817a789e7/users/12712d8a4a80488caaee2e6c8a4a2625/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.780398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=048547025e6f4cbaba52cbc817a789e7, user_id=12712d8a4a80488caaee2e6c8a4a2625, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:19.791465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:19.847600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1103/2223] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:19 2022] PUT /identity/v3/projects/048547025e6f4cbaba52cbc817a789e7/users/12712d8a4a80488caaee2e6c8a4a2625/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:19.859372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583e0f-34fb-4d2d-bada-87685003fcee None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:19.859657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583e0f-34fb-4d2d-bada-87685003fcee None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:19.859937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583e0f-34fb-4d2d-bada-87685003fcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:19.899393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4e583e0f-34fb-4d2d-bada-87685003fcee None None] MFA Rules not processed for user `12712d8a4a80488caaee2e6c8a4a2625`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:19.900599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4e583e0f-34fb-4d2d-bada-87685003fcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:19.993053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1121/2224] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:20.004050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:20.005897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:20.093476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fc14-602c-419e-bf24-4aeaf94cfa7f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:20.098783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:20.099573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:20.099838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:20.100574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:20.113600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:20.114736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:20.131898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1104/2225] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:52:21.670627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63055e2-a129-4ca6-877f-2d96b454777b None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:21.671055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63055e2-a129-4ca6-877f-2d96b454777b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:21.671163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63055e2-a129-4ca6-877f-2d96b454777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:21.692984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f63055e2-a129-4ca6-877f-2d96b454777b None None] MFA Rules not processed for user `12712d8a4a80488caaee2e6c8a4a2625`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:21.694379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f63055e2-a129-4ca6-877f-2d96b454777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:21.711806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1122/2226] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:24.608288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:24.610568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:24.719366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdfb33-e3b1-4710-8f2a-2f4020ccd7c8 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:24.751194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:24.753752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:24.754127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:24.754416 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:24.762119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:24.861077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:24.947609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:24.949638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddcc7b-d333-4ff9-aa40-0af1e0c4afff service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:24.966339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1105/2227] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:52:31.686683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:34.959749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=15 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:37.328949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1ea8-ece6-4321-80b9-659b1e2a293c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:37.329453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1ea8-ece6-4321-80b9-659b1e2a293c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:37.329703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1ea8-ece6-4321-80b9-659b1e2a293c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:37.351743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4aea1ea8-ece6-4321-80b9-659b1e2a293c None None] MFA Rules not processed for user `1dd24e8c0d214a0bb62578f9834a40a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:37.352946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4aea1ea8-ece6-4321-80b9-659b1e2a293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:37.444762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1123/2228] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:40.535389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:40.535872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:40.535965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:40.555521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] MFA Rules not processed for user `07220936c8784339b62f22482264e234`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:40.556707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:40.612072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1106/2229] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:41.527611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b861-6e68-46ac-bce6-738f256d1f47 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:41.528787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b861-6e68-46ac-bce6-738f256d1f47 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:41.529224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b861-6e68-46ac-bce6-738f256d1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:41.555510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-a7e4b861-6e68-46ac-bce6-738f256d1f47 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:41.559072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a7e4b861-6e68-46ac-bce6-738f256d1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:41.662873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1124/2230] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:41.734310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:41.736321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:41.862258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b1e4-09a7-4239-bf27-19494c135331 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:41.882661 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:41.883468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:41.883745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:41.884107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:41.900955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:41.903725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:41.923561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1107/2231] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:52:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:52:44.173339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514d0-785b-49f0-9877-330ddc2cb21c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:44.174449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514d0-785b-49f0-9877-330ddc2cb21c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:44.175058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514d0-785b-49f0-9877-330ddc2cb21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:44.234271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ae7514d0-785b-49f0-9877-330ddc2cb21c None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:44.236130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ae7514d0-785b-49f0-9877-330ddc2cb21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:44.272368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1125/2232] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:44.286072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:44.302422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56a39-a72d-4839-99dd-a2985f606e1d service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:44.316566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:44.317275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:44.317564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:44.317882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:44.359147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:44.360944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0a786-1c58-4f22-82f6-41eaadbbb621 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:44.407852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:44.408505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:44.408946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:44.415174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1108/2233] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:52:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:52:44.454723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] MFA Rules not processed for user `b4d57a4e83ab4310b5b3ba90f43c5ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:44.456386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:44.596981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1126/2234] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:45.522467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a16-c579-4791-bbf0-4f35f49d71cf None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:45.522967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a16-c579-4791-bbf0-4f35f49d71cf None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:45.523247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a16-c579-4791-bbf0-4f35f49d71cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:45.562774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-27ec1a16-c579-4791-bbf0-4f35f49d71cf None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:45.564488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-27ec1a16-c579-4791-bbf0-4f35f49d71cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:45.598511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1109/2235] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 12:52:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:45.621267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:45.645652 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907701de-fb48-43ee-bd61-956480a35bb5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:45.652801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:45.653647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:45.654404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:45.654871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:45.665271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:45.667945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:45.688431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1127/2236] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:52:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:52:46.202325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1d4c-4282-4340-b460-061af031adce None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:46.202616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1d4c-4282-4340-b460-061af031adce None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:46.202955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1d4c-4282-4340-b460-061af031adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:46.293665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ce3a1d4c-4282-4340-b460-061af031adce None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:46.294964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ce3a1d4c-4282-4340-b460-061af031adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:46.318703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1110/2237] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:46.352435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:46.385223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98c7bb-a198-4922-8df0-a8668d5ee715 service swift] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:46.397751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:46.398761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:46.399049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:46.399606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:46.434147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:46.436239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:46.456692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1128/2238] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:52:47.235945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:47.238624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:47.239173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:47.284678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:47.286772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:47.318439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1111/2239] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:47.332631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:47.352234 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e006b2-2eb0-43e1-83f5-03a620f657cf service swift] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:47.360861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:47.361639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:47.362467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:47.362751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:47.383475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:47.385329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63541fd8-ad70-42b1-8fb0-09c54788cdf0 service swift] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:47.398511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1129/2240] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:52:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:52:50.089514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:50.226239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ea94a-17f2-4497-864b-e645f92f5f33 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:50.238770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:50.239662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:50.240095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:50.240529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] PATH_INFO: `/v3/users/12712d8a4a80488caaee2e6c8a4a2625` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:50.243737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=12712d8a4a80488caaee2e6c8a4a2625)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:50.251986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:50.256272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:50.380835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12712d8a4a80488caaee2e6c8a4a2625'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:50.395635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12712d8a4a80488caaee2e6c8a4a2625'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:50.396392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12712d8a4a80488caaee2e6c8a4a2625'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:50.460501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12712d8a4a80488caaee2e6c8a4a2625'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:50.463291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12712d8a4a80488caaee2e6c8a4a2625'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:50.528640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1112/2241] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:52:50 2022] DELETE /identity/v3/users/12712d8a4a80488caaee2e6c8a4a2625 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:52:50.533042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:50.534489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:50.534931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:50.564533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:50.566106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:50.574294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=100 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:50.770258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1130/2242] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:52:50.781372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:50.783222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:50.951296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196ff7a-54c1-4f14-bca4-7bc1b2af4cdc admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:50.956799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:50.957438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:50.957796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:50.958046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:50.971943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:50.973422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42faa183-45a0-44d1-b47a-8ddc3310a4e4 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:50.991875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1113/2243] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:52:51.753513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:51.756054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:51.771736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5d53f-e81c-4f01-9cff-34510e693bda None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:51.780506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:51.781565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:51.782093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:51.782588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] PATH_INFO: `/v3/projects/048547025e6f4cbaba52cbc817a789e7` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:51.784317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048547025e6f4cbaba52cbc817a789e7)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:51.788409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:51.937119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048547025e6f4cbaba52cbc817a789e7', 'request_id': 'req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:51.975512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048547025e6f4cbaba52cbc817a789e7', 'request_id': 'req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:51.993277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048547025e6f4cbaba52cbc817a789e7', 'request_id': 'req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:52.018270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] The token cache is being invalidate because project 048547025e6f4cbaba52cbc817a789e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:52:52.018622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:52.019258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-72f1dbe1-9efb-43e6-bf95-d89e2eadce22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:52:52.026034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1131/2244] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:52:51 2022] DELETE /identity/v3/projects/048547025e6f4cbaba52cbc817a789e7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:52.027822 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.028328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.028417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.064093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:52.065411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:52.172418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1114/2245] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:52.180975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.181263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.181542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.214289 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:52.216472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:52.230308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1132/2246] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:52.231772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.248781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d790500-1564-47ce-b260-209e43d9f949 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.263611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.265035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.267378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.267662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.268403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:52:52.268716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.270437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.303913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1115/2247] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:52:52.309123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.325495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61bbb1-772f-40db-a742-018eaa769909 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.330841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.331753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.332043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.332325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.333022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.335446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.387811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1133/2248] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:52:52 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:52.392759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.417913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5688-fdb4-4c11-b1a6-f8fcce8543eb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.435680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:52.548060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.548741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.548994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.549256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.549957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.551907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.582443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1116/2249] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:52:52 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:52:52.583298 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.595308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28722-b882-455b-8041-c52bd0396730 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.602734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.602734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.602734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.602734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.603636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.606391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.627736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.630169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1134/2250] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:52.649741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f6b9-b396-4e0b-84eb-66c75e9e8b7e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.660445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.661328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.661603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.661956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.662707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.664545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6a97d-993b-402f-ab9a-9118f1e4571a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.683039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.690102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1117/2251] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:52:52.698571 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59789198-196e-458c-9ed1-9c8824aa2319 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:52.703928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:52.705207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.705207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.705511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] PATH_INFO: `/v3/projects/3686c63736c7409eb7cc4170607ca6fb/users/9a73d627ff2b4826b3ab025f5eea27f6/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.736637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3686c63736c7409eb7cc4170607ca6fb, user_id=9a73d627ff2b4826b3ab025f5eea27f6, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:52.741217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:52.770046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1135/2252] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:52:52 2022] PUT /identity/v3/projects/3686c63736c7409eb7cc4170607ca6fb/users/9a73d627ff2b4826b3ab025f5eea27f6/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:52:52.774912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2efe-6d4f-4eb0-a80a-1c7804302ac3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:52.777878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2efe-6d4f-4eb0-a80a-1c7804302ac3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:52.777878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2efe-6d4f-4eb0-a80a-1c7804302ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:52.816633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-26ef2efe-6d4f-4eb0-a80a-1c7804302ac3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:52:52.818301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-26ef2efe-6d4f-4eb0-a80a-1c7804302ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:52.906301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1118/2253] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:52:52.916270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:52:52.920586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:52:52.964274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:52:53.027176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824ee2d-7653-4fa7-9887-be74b59179ac admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:52:53.038501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:52:53.038501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:52:53.038501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:52:53.038501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:52:53.048853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:52:53.050336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:52:53.070001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1136/2254] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:00.822356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:00.824042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:00.825019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:00.873225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] MFA Rules not processed for user `9a73d627ff2b4826b3ab025f5eea27f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:00.887919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:01.023742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1119/2255] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:01.024495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.041435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b9d2f-2b5d-497c-ae19-7d86bb316fe9 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.046375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.047014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.047299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.047576 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.048248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.050251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.078658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1137/2256] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:53:01.082950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.093631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0790b-93bb-4378-90cf-088d0032958b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.111128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:01.158889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.159515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.159797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.160070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.160743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.162727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.190354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1120/2257] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:53:01 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:53:01.191711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.202741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64c33-9701-4ec4-baa1-748a8ad2233a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.207522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.208206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.208591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.208921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.209681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.211378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.222848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1138/2258] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:53:01.227103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.237114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e0d2-6a46-4cb5-8564-e33a8414360d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.241752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.242448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.242728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.242999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] PATH_INFO: `/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c/users/83bdbd5980dd4ee8914583bb1a14e1a4/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.259654 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6569ae0dc9ca48e8bb0be3173e5c063c, user_id=83bdbd5980dd4ee8914583bb1a14e1a4, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.262445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.280881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1121/2259] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:53:01 2022] PUT /identity/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c/users/83bdbd5980dd4ee8914583bb1a14e1a4/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:53:01.289174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.300300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f56ab-1426-4c63-83d8-3286086a5808 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.369328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.370017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.370879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.371900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.372508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.375389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0202a-82eb-429e-9c36-30e95484ba0d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.390162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1139/2260] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:53:01.394023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:01.411358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f632a-8051-4927-9740-319e35cf3587 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:01.416377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:01.417174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:01.417526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:01.417923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] PATH_INFO: `/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c/users/83bdbd5980dd4ee8914583bb1a14e1a4/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:01.419937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6569ae0dc9ca48e8bb0be3173e5c063c, user_id=83bdbd5980dd4ee8914583bb1a14e1a4, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:01.422706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db9636-1596-4c46-ba58-9788c1ad919d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:01.438996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1122/2261] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:53:01 2022] PUT /identity/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c/users/83bdbd5980dd4ee8914583bb1a14e1a4/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:53:04.543164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:04.543689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:04.543689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:04.545409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1140/2262] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:53:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:53:04.551867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376f1c-68c4-47b4-9b4c-28e611d73307 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:04.552401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376f1c-68c4-47b4-9b4c-28e611d73307 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:04.553085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376f1c-68c4-47b4-9b4c-28e611d73307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:04.581978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-61376f1c-68c4-47b4-9b4c-28e611d73307 None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:04.583781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-61376f1c-68c4-47b4-9b4c-28e611d73307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:04.695005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1123/2263] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:04.708158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:04.709985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:04.844311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9e650-5b1d-49b0-8c60-8e90266ca108 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:04.850478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:04.851381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:04.851823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:04.852255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:04.875737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:04.877258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd250c30-cc6f-40bb-8246-fdc4fc7ebb98 service glance] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:04.894159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1141/2264] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:05.006093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b09bb6-1cea-404f-8abf-b2e9c3512ba8 None None] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.006423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b09bb6-1cea-404f-8abf-b2e9c3512ba8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.006742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b09bb6-1cea-404f-8abf-b2e9c3512ba8 None None] PATH_INFO: `/` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.008468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1124/2265] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:53:05.014708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a9b96-6876-45d5-a738-bb4e0e3b027c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.015107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a9b96-6876-45d5-a738-bb4e0e3b027c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.015634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a9b96-6876-45d5-a738-bb4e0e3b027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.043467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-713a9b96-6876-45d5-a738-bb4e0e3b027c None None] MFA Rules not processed for user `c22269afbdf541d4bedbef2cf8503521`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:05.044837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-713a9b96-6876-45d5-a738-bb4e0e3b027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:05.134090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:05.163391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1142/2266] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:05.174096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b09bb6-1cea-404f-8abf-b2e9c3512ba8 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:05.194553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b09bb6-1cea-404f-8abf-b2e9c3512ba8 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:05.198399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:05.199030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.199315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.199593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.208831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:05.210251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea53874-f2c2-4038-8cb2-69a2bead82a2 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:05.219656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1125/2267] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:05.350372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61e7e-6ae9-4561-b464-0c20d41cc7dd None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.351145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61e7e-6ae9-4561-b464-0c20d41cc7dd None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.351145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61e7e-6ae9-4561-b464-0c20d41cc7dd None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.352658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1143/2268] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 12:53:05.361796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0802c38-407f-4b37-80c3-16acc4582f2b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.362105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0802c38-407f-4b37-80c3-16acc4582f2b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.362387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0802c38-407f-4b37-80c3-16acc4582f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.396919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e0802c38-407f-4b37-80c3-16acc4582f2b None None] MFA Rules not processed for user `8a4c233f6ff1442b8e08fbe6400ecf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:05.398187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e0802c38-407f-4b37-80c3-16acc4582f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:05.408862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1126/2269] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:05.424288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df61e7e-6ae9-4561-b464-0c20d41cc7dd None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:05.436410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df61e7e-6ae9-4561-b464-0c20d41cc7dd None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:05.442129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:05.443459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:05.443821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:05.444178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:05.465836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:05.468726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:05.512408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1144/2270] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:07.848923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762b49a-535d-4832-88bf-4ad998daf0a6 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:07.849401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762b49a-535d-4832-88bf-4ad998daf0a6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:07.849495 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762b49a-535d-4832-88bf-4ad998daf0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:07.900067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6762b49a-535d-4832-88bf-4ad998daf0a6 None None] MFA Rules not processed for user `83bdbd5980dd4ee8914583bb1a14e1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:07.901478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6762b49a-535d-4832-88bf-4ad998daf0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:07.941045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:07.988877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1127/2271] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:08.001002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:08.010503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514172f-8f79-475c-9930-5d3b2839b4b0 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:08.014330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:08.015177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:08.015659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:08.016127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:08.080073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:08.081796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:08.093521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1145/2272] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:08.264438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:08.265046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:08.265540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:08.290114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:08.292233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:08.364766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1128/2273] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:08.379882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:08.381206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:08.471998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90a9de-287f-4f40-9b91-8110442f0a46 service glance] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:08.477418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:08.478096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:08.478380 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:08.478695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:08.490239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:08.492651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4c3cc-23af-4593-821d-f29bbf0cfd90 service swift] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:08.504033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1146/2274] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:53:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:53:09.702376 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:09.713533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4276060-3b86-4c6b-b46c-380c3f93d188 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:09.725266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:09.726263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:09.726621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:09.726958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:09.732106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:09.816409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:09.818271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:09.841536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1129/2275] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:10.417314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbceb-f43f-4c90-b77e-0324408142a4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:10.417811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbceb-f43f-4c90-b77e-0324408142a4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:10.417932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbceb-f43f-4c90-b77e-0324408142a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:10.443980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8a2fbceb-f43f-4c90-b77e-0324408142a4 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:10.445245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8a2fbceb-f43f-4c90-b77e-0324408142a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:10.455928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1147/2276] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:10.479699 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:10.494290 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2a356-f0a2-438a-91f2-a15bed2a9ff7 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:10.499363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:10.499985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:10.500265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:10.500660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:10.510230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:10.511956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:10.524754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1130/2277] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:53:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:53:20.436631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=75 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:20.520318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=75 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:20.593127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17374d-1824-44f0-b3c5-e1fbe02bc2eb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:20.593752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17374d-1824-44f0-b3c5-e1fbe02bc2eb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:20.594076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17374d-1824-44f0-b3c5-e1fbe02bc2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:20.615490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ad17374d-1824-44f0-b3c5-e1fbe02bc2eb None None] MFA Rules not processed for user `9a73d627ff2b4826b3ab025f5eea27f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:20.616908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ad17374d-1824-44f0-b3c5-e1fbe02bc2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:20.624957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1148/2278] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:26.785102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:26.786781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:26.871031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76ec22-4080-429b-89b3-ee9b939df5d7 service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:26.875651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:26.876362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:26.876719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:26.877193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:26.879636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:26.899114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:26.901799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:26.913598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1131/2279] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:53:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:53:28.795276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d37ac-b350-41cd-aff2-58b73dde0cc4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:28.796288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d37ac-b350-41cd-aff2-58b73dde0cc4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:28.796755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d37ac-b350-41cd-aff2-58b73dde0cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:28.818461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b93d37ac-b350-41cd-aff2-58b73dde0cc4 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:28.819760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b93d37ac-b350-41cd-aff2-58b73dde0cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:28.827727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1149/2280] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:53:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:28.840998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:28.853067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb923a3-298f-4bc8-830c-b3d1a5738ab7 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:28.858297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:28.858935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:28.859215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:28.859491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:28.869463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:28.871103 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:28.881880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1132/2281] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:53:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:53:38.811953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:38.876850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=38 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:48.209412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:48.209928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:48.209928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:48.230392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:53:48.231690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:53:48.239077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1150/2282] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 12:53:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:53:48.264437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:53:48.275396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2b2b1-48b1-4f0f-8eb4-36acae04cf8a service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:53:48.280432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:53:48.281790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:53:48.282128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:53:48.282436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:53:48.291205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:53:48.292789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479660a3-e904-4e27-9b23-edea4199bae8 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:53:48.303211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1133/2283] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:53:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:53:58.224552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:53:58.300134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:54:04.305965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:54:04.315282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255c1b1-49f5-4064-8d30-ee1899371d4c None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:54:04.318818 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:54:04.319376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:54:04.319658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:54:04.320066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:54:04.330565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:54:04.332268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b90e1c-6eef-44c2-b57f-7f2b677b972c service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:54:04.341978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1151/2284] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:54:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:54:14.337793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:54:22.041279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5110-1b5e-42f8-97c2-7b8ad9dc5120 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:54:22.041803 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5110-1b5e-42f8-97c2-7b8ad9dc5120 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:54:22.041967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5110-1b5e-42f8-97c2-7b8ad9dc5120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:54:22.063828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f38f5110-1b5e-42f8-97c2-7b8ad9dc5120 None None] MFA Rules not processed for user `b4d57a4e83ab4310b5b3ba90f43c5ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:54:22.065063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f38f5110-1b5e-42f8-97c2-7b8ad9dc5120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:54:22.119382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1134/2285] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:54:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:54:32.112466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=25 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:54:38.098399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:54:38.098964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:54:38.098964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:54:38.123605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] MFA Rules not processed for user `1dd24e8c0d214a0bb62578f9834a40a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:54:38.124789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:54:38.182127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1152/2286] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:54:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:54:48.175486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=25 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:54:57.196533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:54:57.197126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:54:57.197374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:54:57.218707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] MFA Rules not processed for user `e7ee2c3480f8483bb76ecc686215720b`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:54:57.220833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:54:57.299096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1135/2287] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:54:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:55:07.286048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=25 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:12.781672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:12.783136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:12.794933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e171022-e35c-425d-9cab-5ef2a5ea3454 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:12.798561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:12.799179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:12.799461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:12.799740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:12.801466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:12.815784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:12.817210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:12.828927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1153/2288] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:14.004768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:14.015232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e340-7890-41c5-8bd5-769906261a09 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:14.018917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:14.019540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:14.019827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:14.020098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:14.022870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:14.036984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:14.038457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:14.049362 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1136/2289] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:22.823939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:24.044563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:28.959464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:28.960906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:29.018897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f83a3-b173-48d5-ac98-662f725a7b60 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:29.023158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:29.023769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:29.024049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:29.024325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:29.026096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:29.036672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:29.038111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:29.046843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1154/2290] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:32.120341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:32.130482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bad7a1-75ce-47aa-9864-3a242bd69f40 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:32.134042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:32.134644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:32.134921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:32.135189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:32.136899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:32.148013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:32.149414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318f077-cb76-4aeb-aee5-82901d86b178 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:32.158133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1137/2291] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:39.042942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=26 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:42.154152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:44.103749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:44.105523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:44.142199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cde2bd-264f-4ebd-b444-6555a16ab26e service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:44.155291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:44.156743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:44.156983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:44.157618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:44.160801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:44.177390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:44.181089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:44.245880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1155/2292] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:45.691115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:45.692047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:45.692476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:45.743018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] MFA Rules not processed for user `07220936c8784339b62f22482264e234`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:55:45.744259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:45.816147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1138/2293] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:55:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:55:46.643716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:46.653506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b7b9a-8ba2-4abb-b96e-ac8fe792da7d service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:46.657140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:46.657751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:46.658064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:46.658342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:46.660094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:46.670317 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:46.671770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa415a-7a31-422c-80f5-9771f25eb848 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:46.680088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1156/2294] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:47.232998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:47.235335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:47.251285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61f0af-4b9a-4d35-94d8-c3106c4a942e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:47.256703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:47.257337 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:47.257618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:47.257927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:47.259723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:47.272242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:47.274226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:47.297254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1139/2295] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:48.234190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:48.234737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:48.234843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:48.254444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] MFA Rules not processed for user `b4d57a4e83ab4310b5b3ba90f43c5ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:55:48.255653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:48.262760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1157/2296] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:55:51.209586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:51.223777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecadebe-6cc0-4c5f-9429-1ef8b8d9c215 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:51.228321 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:51.228963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:51.229241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:51.229508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:51.231569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:51.242069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:51.243618 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:51.253724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1140/2297] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:55:58.248882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=15 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:55:59.239333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:55:59.240736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:59.256589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaeb4c-ee55-4250-bab3-e2d122470ede None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:55:59.262016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:55:59.262661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:55:59.262943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:55:59.263220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:55:59.264938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:55:59.275748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:55:59.277346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5888-0775-4b3a-9fdb-588594b5baec service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:55:59.288119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1158/2298] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:55:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:00.752134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:00.761452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8e739-eeaa-4770-a0d1-ba80e2d24023 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:00.765214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:00.765849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:00.766240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:00.766519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:00.768527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:00.834798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:00.836221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee2fd-6ed9-4514-b59d-2c4b0a66acd5 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:00.845956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1141/2299] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:04.327807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe465-72ae-48fd-ac86-914f87cf8782 None None] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.328340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe465-72ae-48fd-ac86-914f87cf8782 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.328340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe465-72ae-48fd-ac86-914f87cf8782 None None] PATH_INFO: `/` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.329695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1159/2300] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 29 12:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 12:56:04.348858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.349505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.349719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.373284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] MFA Rules not processed for user `b74de7e13793482882a0097940e339c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:04.374523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:04.384478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1142/2301] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 29 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:04.417227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe465-72ae-48fd-ac86-914f87cf8782 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:04.460917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:04.462442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:04.483940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe465-72ae-48fd-ac86-914f87cf8782 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:04.492901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bdafe-3283-4f78-93d3-6a1d366e3b32 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:04.504563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:04.505302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.505882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.506343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] PATH_INFO: `/v3/users/1dd24e8c0d214a0bb62578f9834a40a2` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.507794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:04.508032 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd24e8c0d214a0bb62578f9834a40a2)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:04.508727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.509106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.509492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.510557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:04.520775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:04.523239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:04.538633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1160/2302] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:04.567078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd24e8c0d214a0bb62578f9834a40a2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.576101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd24e8c0d214a0bb62578f9834a40a2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.576833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd24e8c0d214a0bb62578f9834a40a2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.582998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:04.593484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd24e8c0d214a0bb62578f9834a40a2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.594231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-d45d8a84-34ea-4cfb-b252-b67ee6957380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd24e8c0d214a0bb62578f9834a40a2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.607241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1143/2303] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:04 2022] DELETE /identity/v3/users/1dd24e8c0d214a0bb62578f9834a40a2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:04.790886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:04.802405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae84bc-0069-4688-9541-eb29cf654a8c service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:04.860093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:04.860745 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.861028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.861308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] PATH_INFO: `/v3/projects/9cdcbcec39334733a5fa29ddc40c7a2d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.862466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdcbcec39334733a5fa29ddc40c7a2d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:04.864392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:04.909201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdcbcec39334733a5fa29ddc40c7a2d', 'request_id': 'req-58770430-a069-4ebc-9b88-dfb9adcaf3c3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.914856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdcbcec39334733a5fa29ddc40c7a2d', 'request_id': 'req-58770430-a069-4ebc-9b88-dfb9adcaf3c3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.922276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdcbcec39334733a5fa29ddc40c7a2d', 'request_id': 'req-58770430-a069-4ebc-9b88-dfb9adcaf3c3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.936600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] The token cache is being invalidate because project 9cdcbcec39334733a5fa29ddc40c7a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:04.936889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.937537 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-58770430-a069-4ebc-9b88-dfb9adcaf3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:04.939150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1161/2304] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:04 2022] DELETE /identity/v3/projects/9cdcbcec39334733a5fa29ddc40c7a2d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:04.944864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:04.945155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:04.945430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:04.966915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:04.968102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:05.029266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1144/2305] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:56:05.037083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.037375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.037653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.059937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:05.061186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:05.069879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1162/2306] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:05.074219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.084407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e37b40-ef9c-4f74-8012-ae42aac46ee3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.089576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.090226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.090510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.090783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.091514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:56:05.091790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.093385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.106291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1145/2307] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:56:05.110843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.122164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce2301-3752-4f6c-84bc-4ed81e183fc1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.126910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.127556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.127841 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.128117 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.128812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.131313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.159919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1163/2308] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:05 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:05.164308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.175227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23594ada-6392-465d-8e18-dc737720a435 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.245635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.246295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.246580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.246858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.247778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.249870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.273738 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1146/2309] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:05 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:05.277507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.289844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71bda1-1ec7-409b-b406-9d237b2c0744 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.294912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.295655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.296206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.296547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.298152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.299743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.312408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1164/2310] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:56:05.316869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.327966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8e2ff-6a50-48b7-9a63-e6851cdc44a5 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.334250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.334881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.335161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.335435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.336306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.338227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0208f1-56f4-4b2f-834e-9be3e8c485b1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.352792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1147/2311] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:05.356976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.367496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc87240-a713-4c5d-a7c4-69bacdc469dd admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.372291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.372960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.373245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.373545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] PATH_INFO: `/v3/projects/c405cac30de74210a1a7ab4d8d0b57e7/users/67f6066f82e94c7384f7db33fe62b6f8/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.390320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c405cac30de74210a1a7ab4d8d0b57e7, user_id=67f6066f82e94c7384f7db33fe62b6f8, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.392998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.403435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:05.409112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1165/2312] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:05 2022] PUT /identity/v3/projects/c405cac30de74210a1a7ab4d8d0b57e7/users/67f6066f82e94c7384f7db33fe62b6f8/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:56:05.420886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.421241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.421619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.451295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] MFA Rules not processed for user `67f6066f82e94c7384f7db33fe62b6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:05.452627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:05.486289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:05.523160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1148/2313] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:05.534102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:05.538161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:05.602863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e0900-4cff-4a13-acae-741f98dc4594 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:05.606547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:05.607187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:05.607472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:05.607747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:05.617811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:05.619284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:05.630182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1166/2314] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:06.685950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:06.687538 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:06.768698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edac1f-454d-468c-9ff2-a22ec09d813f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:06.776602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:06.777232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:06.777511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:06.777891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:06.781751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:06.794455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:06.796109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4789-4858-4ed4-bdce-c3aef3db53c2 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:06.806995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1149/2315] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:14.468295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:14.544441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee8aef-3df0-453d-bead-3ab32a9820b9 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:14.554191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:14.554973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:14.555402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:14.555509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] PATH_INFO: `/v3/users/67f6066f82e94c7384f7db33fe62b6f8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:14.556721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f6066f82e94c7384f7db33fe62b6f8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:14.559282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:14.652880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f6066f82e94c7384f7db33fe62b6f8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:14.654086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f6066f82e94c7384f7db33fe62b6f8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:14.667259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f6066f82e94c7384f7db33fe62b6f8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:14.686986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f6066f82e94c7384f7db33fe62b6f8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:14.701217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f6066f82e94c7384f7db33fe62b6f8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:14.711147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1167/2316] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:14 2022] DELETE /identity/v3/users/67f6066f82e94c7384f7db33fe62b6f8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:14.715960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fcd91-048b-4789-bfcd-e0dd3f09137c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:14.716492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fcd91-048b-4789-bfcd-e0dd3f09137c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:14.716492 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fcd91-048b-4789-bfcd-e0dd3f09137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:14.751462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-138fcd91-048b-4789-bfcd-e0dd3f09137c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:14.753189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-138fcd91-048b-4789-bfcd-e0dd3f09137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:14.851060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1150/2317] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:14.860682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:14.862264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:14.899452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:14.971824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3bda6-4529-484f-8015-3cfda80fa98d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:14.976309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:14.977812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:14.978612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:14.978612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:14.995165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:15.007188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:15.022124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1168/2318] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:15.953993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:15.954982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:15.955263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:15.982090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:15.983690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.060668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.062259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.068299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1151/2319] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:16.110399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75462f49-c368-4042-b848-4b407ff07d3c service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.130504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.131174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.132386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.132718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] PATH_INFO: `/v3/projects/c405cac30de74210a1a7ab4d8d0b57e7` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.145254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c405cac30de74210a1a7ab4d8d0b57e7)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.147211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:16.158059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.160247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.169078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:16.282241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0932b36-028b-4396-8d86-421b3b222162 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.304759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c405cac30de74210a1a7ab4d8d0b57e7', 'request_id': 'req-10afd540-4098-451c-b483-86d62415e07a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:16.319896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c405cac30de74210a1a7ab4d8d0b57e7', 'request_id': 'req-10afd540-4098-451c-b483-86d62415e07a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:16.342505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c405cac30de74210a1a7ab4d8d0b57e7', 'request_id': 'req-10afd540-4098-451c-b483-86d62415e07a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:16.365752 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] The token cache is being invalidate because project c405cac30de74210a1a7ab4d8d0b57e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:16.368972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:16.371379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-10afd540-4098-451c-b483-86d62415e07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:16.376015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1169/2320] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:16 2022] DELETE /identity/v3/projects/c405cac30de74210a1a7ab4d8d0b57e7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:16.384435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7d40-8949-4249-bd51-f10eaa05f06e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.387883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7d40-8949-4249-bd51-f10eaa05f06e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.388178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7d40-8949-4249-bd51-f10eaa05f06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.444950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-735a7d40-8949-4249-bd51-f10eaa05f06e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:16.449877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-735a7d40-8949-4249-bd51-f10eaa05f06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.455693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.456468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.457392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.457767 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.459877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.568124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1170/2321] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:16.576249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.576959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.577388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.577508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.579251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:16.589958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1152/2322] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:56:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:56:16.604361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.606877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.622575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:16.622575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.641106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1171/2323] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:16.644960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.660973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846c32b-1966-49fb-952a-ec0cffcfac87 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.674476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.675110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.675464 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.675735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.676434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:56:16.676748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.678479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:16.707948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:16.712036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1172/2324] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:56:16.713169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.734061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848facb5-e431-421c-ad6c-a44f26ee8184 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.741316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.741999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.742283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.742558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.743299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.747624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:16.770448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ef0de-93b9-49d0-9a05-e32f97cab7c2 service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.784537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.786833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.787670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.788082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.794600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:16.803917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1173/2325] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:16 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:16.811719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:16.818959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.820719 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:16.824814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:16.840277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1153/2326] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:16.948209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69e89-c28e-4304-8b52-a5c9f517b8f9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:16.955444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:16.957038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:16.957375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:16.957659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:16.959418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:16.963647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.001148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1174/2327] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:16 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:17.005298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.022709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef0d5-dcc8-4d9d-8119-b59bf57076f1 service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.037063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.038346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.038634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.038908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.039624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.041310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.064253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1154/2328] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:17.069286 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.084232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384496-5e3f-4831-a8e7-30fcb0dfc9ad admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.097872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.100715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.103504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.104409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.108897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.113422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1e6e8-5c2b-489f-bba2-7127e2ebca16 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.150094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1175/2329] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:17.156392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.167553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c19f6-3115-444d-b2d8-a7bff8ec86fd admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.186039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.187009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.187457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.188350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] PATH_INFO: `/v3/projects/129f92af91de42659df59fa0db63995e/users/5ac7d785838d4fab97f24e1c06a143f9/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.230972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129f92af91de42659df59fa0db63995e, user_id=5ac7d785838d4fab97f24e1c06a143f9, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.233749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.261744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1155/2330] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:17 2022] PUT /identity/v3/projects/129f92af91de42659df59fa0db63995e/users/5ac7d785838d4fab97f24e1c06a143f9/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:17.272382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.272750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.273097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.326816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] MFA Rules not processed for user `5ac7d785838d4fab97f24e1c06a143f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:17.328201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:17.475119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1176/2331] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:17.477294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.674874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962d372-fa6c-4f9c-a46e-e6461d7b34f8 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.693740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.694437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.694718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.697570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.698322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.700614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.753063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1156/2332] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:17 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:17.757805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.773026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1575756-b3ed-41b5-8021-9fe08ccef0a3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.872534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.873790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.874359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.874874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.875814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.878084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.915441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1177/2333] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:17 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:17.919281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.938037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153514f8-1f46-4cf2-97fb-f098bbaf7db4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.949486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.951294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.951832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.952394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:17.953651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:17.955915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:17.971054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1157/2334] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:56:17.975251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:17.982438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:17.987724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b204-95ee-4053-85ea-cd12d5524f22 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:17.994991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:17.995840 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:17.996453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:17.996831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] PATH_INFO: `/v3/projects/f94bf14389f74f39a6e4154106100246/users/90e6780c290c4a4b9580a8731b129658/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:18.017954 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94bf14389f74f39a6e4154106100246, user_id=90e6780c290c4a4b9580a8731b129658, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:18.020901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:18.043846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1178/2335] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:17 2022] PUT /identity/v3/projects/f94bf14389f74f39a6e4154106100246/users/90e6780c290c4a4b9580a8731b129658/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:18.045610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:18.062902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6871edf-76f3-4a4c-85fd-c537ad74d771 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:18.159147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:18.160701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:18.161256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:18.161760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:18.162943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:18.164858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85966725-5fc3-44f1-b78f-43b1ec3fb883 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:18.180404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1158/2336] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:18.181984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:18.196149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b4ae2-d10e-4ae6-aa8d-25babd40bef9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:18.220907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:18.222036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:18.222935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:18.223484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] PATH_INFO: `/v3/projects/f94bf14389f74f39a6e4154106100246/users/90e6780c290c4a4b9580a8731b129658/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:18.225711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94bf14389f74f39a6e4154106100246, user_id=90e6780c290c4a4b9580a8731b129658, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:18.232516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:18.261985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1179/2337] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:18 2022] PUT /identity/v3/projects/f94bf14389f74f39a6e4154106100246/users/90e6780c290c4a4b9580a8731b129658/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:56:18.271468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:18.271879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:18.272785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:18.308501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:18.322980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] MFA Rules not processed for user `90e6780c290c4a4b9580a8731b129658`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:18.324406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:18.458795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1159/2338] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:18.481644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:18.483532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:18.676582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f794be-fe82-489d-9130-d3e596b9de86 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:18.689718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:18.689718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:18.690037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:18.690989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:18.715906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:18.718612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3091a60-2cee-45c8-9eee-86e9f650211e service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:18.745967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1180/2339] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:18.887397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:18.889470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:18.919514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc74e-aab3-4e90-a178-e7c0d36a568a None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:18.927296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:18.930199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:18.930560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:18.930847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:19.072861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:19.076106 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317397f-2b4d-449a-af82-4dd883cf2117 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:19.107466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1160/2340] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:19.391376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:19.391887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:19.391984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:19.413264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] MFA Rules not processed for user `03f312d5c7334ccc82be17c5dfe29aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:19.414516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:19.488578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1181/2341] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:29.078743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:29.079307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:29.079673 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:29.086476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:29.113242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:29.120799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:29.217745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1161/2342] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:29.218126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b16d4-9281-47fa-8740-f315e9f0e6c4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:29.224875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:29.225782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:29.226630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:29.227119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] PATH_INFO: `/v3/users/5ac7d785838d4fab97f24e1c06a143f9` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:29.230366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:29.233187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:29.234478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac7d785838d4fab97f24e1c06a143f9)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:29.237272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:29.264110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:29.327728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:29.361968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac7d785838d4fab97f24e1c06a143f9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:29.364016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac7d785838d4fab97f24e1c06a143f9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:29.376383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac7d785838d4fab97f24e1c06a143f9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:29.381213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68bc2b-9512-4b98-a836-e512c32df657 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:29.386755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:29.387375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:29.387657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:29.388062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:29.395165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac7d785838d4fab97f24e1c06a143f9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:29.397163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:29.398993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488f1f1-e1f9-4177-9645-7823915c5655 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:29.407618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac7d785838d4fab97f24e1c06a143f9'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:29.410115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1162/2343] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:56:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 12:56:29.418855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1182/2344] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:29 2022] DELETE /identity/v3/users/5ac7d785838d4fab97f24e1c06a143f9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:29.421781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:29.422152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:29.422489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:29.454665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:29.455881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:29.537004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1163/2345] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:29.556433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:29.557977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:29.658054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad99-18e4-4897-b7b7-5cb9d44718de admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:29.663540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:29.665126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:29.665769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:29.666269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:29.677801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:29.679482 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:29.691241 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1183/2346] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:56:30.070860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:30.072550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:30.086100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee118b83-e811-45ab-a7ea-4e4c090c223d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:30.093640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:30.094414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:30.094911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:30.095296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] PATH_INFO: `/v3/projects/129f92af91de42659df59fa0db63995e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:30.097755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129f92af91de42659df59fa0db63995e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:30.100729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:30.154043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129f92af91de42659df59fa0db63995e', 'request_id': 'req-57c67a9c-46c2-469a-93f9-a09c51f16022'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.166926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129f92af91de42659df59fa0db63995e', 'request_id': 'req-57c67a9c-46c2-469a-93f9-a09c51f16022'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.176392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129f92af91de42659df59fa0db63995e', 'request_id': 'req-57c67a9c-46c2-469a-93f9-a09c51f16022'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.188303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] The token cache is being invalidate because project 129f92af91de42659df59fa0db63995e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:30.188501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.189171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.191070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1164/2347] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:30 2022] DELETE /identity/v3/projects/129f92af91de42659df59fa0db63995e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:30.194949 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:30.205663 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a65c1-2410-4cf5-9106-756f24db7a9b service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:30.288754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:30.289400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:30.289680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:30.289996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] PATH_INFO: `/v3/users/90e6780c290c4a4b9580a8731b129658` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:30.291165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e6780c290c4a4b9580a8731b129658)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:30.293094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:30.339716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:30.351390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e6780c290c4a4b9580a8731b129658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.352179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e6780c290c4a4b9580a8731b129658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.366422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e6780c290c4a4b9580a8731b129658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.384921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e6780c290c4a4b9580a8731b129658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.400339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b97f9f97-dae6-48bd-bbad-8005fa509ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e6780c290c4a4b9580a8731b129658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:30.415395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1184/2348] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:30 2022] DELETE /identity/v3/users/90e6780c290c4a4b9580a8731b129658 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:30.923629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:30.927687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:30.958238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c67a9c-46c2-469a-93f9-a09c51f16022 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.021807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:31.033430 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.034720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.035183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.035574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] PATH_INFO: `/v3/projects/f94bf14389f74f39a6e4154106100246` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.037557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94bf14389f74f39a6e4154106100246)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.039502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.129473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94bf14389f74f39a6e4154106100246', 'request_id': 'req-1348cd5b-5664-4de6-aa3b-ab4ab6509241'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:31.136540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94bf14389f74f39a6e4154106100246', 'request_id': 'req-1348cd5b-5664-4de6-aa3b-ab4ab6509241'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:31.146846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94bf14389f74f39a6e4154106100246', 'request_id': 'req-1348cd5b-5664-4de6-aa3b-ab4ab6509241'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:31.160557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] The token cache is being invalidate because project f94bf14389f74f39a6e4154106100246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:31.160881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:31.161512 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1348cd5b-5664-4de6-aa3b-ab4ab6509241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:31.163979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1165/2349] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:30 2022] DELETE /identity/v3/projects/f94bf14389f74f39a6e4154106100246 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:31.169053 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.169321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.169674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.194115 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:31.195776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:31.267548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1185/2350] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:56:31.275510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.275810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.276090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.297194 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:31.298462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:31.307386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1166/2351] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:31.311823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.322962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474a8e-544c-4cc1-a25e-32d8751f9a9b None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.327616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.328253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.328536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.328810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.329608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:56:31.329956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.331626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.344507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1186/2352] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:56:31.349293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.359578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14107cd-d30a-45cc-91ad-e480ad79b637 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.365024 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.365659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.365980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.366255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.366935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.368864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.400193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1167/2353] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:31.404494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.415065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11377e-dde5-4dca-9960-c8c3a551657a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.476683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.477312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.477596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.477905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.478578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.480533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.507344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1187/2354] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:31.508596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.519464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1c28-928f-4503-a6bd-4771817188ff admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.524157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.524821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.525128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.525381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.526123 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.527724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.538734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1168/2355] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:56:31.542784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.553759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204bfa-b35f-488d-a938-7b325e6e839a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.558519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.559128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.559415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.559692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.560389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.562026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1da0b7-b784-4664-98c9-f21dd2b8cf18 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.574549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1188/2356] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:31.578344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.625699 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae87a8a-4492-4c16-ac7d-46588842f72a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.631386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.632122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.632405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.632682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] PATH_INFO: `/v3/projects/fb9ba1eae2f546cb9da67b8241ea9d1a/users/95cc4967b70a46318bb73ecc24f20358/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.649020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9ba1eae2f546cb9da67b8241ea9d1a, user_id=95cc4967b70a46318bb73ecc24f20358, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.652234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.677936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1169/2357] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:31 2022] PUT /identity/v3/projects/fb9ba1eae2f546cb9da67b8241ea9d1a/users/95cc4967b70a46318bb73ecc24f20358/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:56:31.679666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.679958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.680563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.709546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] MFA Rules not processed for user `95cc4967b70a46318bb73ecc24f20358`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:31.710823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:31.720802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:31.791284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1189/2358] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:31.795674 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.917696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739b476-8a74-4c08-bb71-72b42be0fa94 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:31.924132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:31.924781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:31.934835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:31.936419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:31.937164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:31.940624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:31.973694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1170/2359] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:31.977964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:31.989182 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ee1a0-53e5-4563-8924-ed0f7fac001f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.087201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.087566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.087846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.088120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.088935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.091424 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.128859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1190/2360] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:31 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:32.130192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.135736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:32.141344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5a7ee-2db6-430e-ae11-f2d0a48dc1d6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.149119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.150176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.150545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.150945 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.152136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.154753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.170300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1171/2361] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:32.171693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.182914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc3de-e14c-4b1d-8f71-f21d32645ff2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.190475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.191446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.191899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.192545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.193635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.195309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e0666-c8fd-4e81-bf5d-9182745ab88d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.208499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1191/2362] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:32.212783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.224509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f5ae2-5e1c-4d3d-8c0c-fa10226184fd admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.229266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.230117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.230462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.230806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] PATH_INFO: `/v3/projects/1f2aa4b08fc14682b0ed494236c3a07d/users/6e6fdaccd66446f2a4e06ce5701a3912/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.247135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2aa4b08fc14682b0ed494236c3a07d, user_id=6e6fdaccd66446f2a4e06ce5701a3912, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.250020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.274861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1172/2363] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:32 2022] PUT /identity/v3/projects/1f2aa4b08fc14682b0ed494236c3a07d/users/6e6fdaccd66446f2a4e06ce5701a3912/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:32.281993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.282364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.282717 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.314702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] MFA Rules not processed for user `6e6fdaccd66446f2a4e06ce5701a3912`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:32.316587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:32.397983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1192/2364] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:32.402010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.505323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0b339-84ed-4549-a98c-dbc1c9ff4860 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.513316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.514235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.514683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.515014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.519919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.523129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.561998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1173/2365] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:32 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:56:32.563641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.577905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3dffd-3ab4-46ab-ab9e-10876b9b9f94 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.602207 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:32.662517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.663155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.663603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.663889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.664644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.666696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.696903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1193/2366] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:32 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:56:32.698238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.709941 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ce49-de3f-4b0d-93a9-293acf8419b9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.715142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.715524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.715881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.716221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.716993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.718683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.737155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1174/2367] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:32.738685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.752541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47f55c-2e52-48f0-bf53-c250ea6d73aa admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.759539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.760847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.761190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.761547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] PATH_INFO: `/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a/users/fde260f1a83d480887a63483c7a8bf45/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.779833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6207fd11ef549cc83fbfe5e61d0e36a, user_id=fde260f1a83d480887a63483c7a8bf45, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.782894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.799068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1194/2368] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:32 2022] PUT /identity/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a/users/fde260f1a83d480887a63483c7a8bf45/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:56:32.803202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.819561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f4142-c5df-4777-971e-99554fda9a6f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.902315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.903030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.903386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.903734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.905268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.907549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f6103-3dd2-4fa5-a391-d76daa910b7a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.926390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1175/2369] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:32.930205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:32.945283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd12433-379b-437d-bd98-b44ec79c5d72 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:32.950555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:32.951340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.951941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.952322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] PATH_INFO: `/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a/users/fde260f1a83d480887a63483c7a8bf45/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:32.954519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6207fd11ef549cc83fbfe5e61d0e36a, user_id=fde260f1a83d480887a63483c7a8bf45, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:32.958185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:32.975154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1195/2370] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:32 2022] PUT /identity/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a/users/fde260f1a83d480887a63483c7a8bf45/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:32.985379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:32.985733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:32.986130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:33.019272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] MFA Rules not processed for user `fde260f1a83d480887a63483c7a8bf45`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:33.020655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:33.051728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:33.103454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1176/2371] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:56:33.120157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:33.121762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:33.217386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f7b5-dcd4-487d-afa8-cd93c455708e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:33.223527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:33.224334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:33.224616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:33.224891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:33.238562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:33.240253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0020f-9736-4d38-9fc1-79df9693bc17 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:33.251367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1196/2372] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:34.108414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:34.132589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c774ef-c17e-4ce8-bd65-1635e19c182d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:34.137936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:34.138653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:34.139012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:34.139363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:34.247453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:34.249853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:34.285879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1177/2373] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:36.612967 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:36.613490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:36.613490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:36.637515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:36.639303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:36.743375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1197/2374] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:36.757048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:36.758603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:36.845165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107edb97-429b-4341-ae61-e8ed1ea8812e service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:36.850488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:36.851131 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:36.851410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:36.851685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:36.861390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:36.863135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:36.872841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1178/2375] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:56:38.926094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:38.927606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:38.952975 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:38.954688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:38.966977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:38.972365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3604-cdba-4181-9ef6-c04069a532fe service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:38.978624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:38.979426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:38.979856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:38.981973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:39.074578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb499533-d128-4120-a6e1-3ce14de64aa0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:39.080522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:39.081428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:39.081784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:39.082161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] PATH_INFO: `/v3/users/9a73d627ff2b4826b3ab025f5eea27f6` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:39.083625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a73d627ff2b4826b3ab025f5eea27f6)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:39.086558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:39.093279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:39.108922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:39.110783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:39.136959 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1179/2376] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:39.169548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a73d627ff2b4826b3ab025f5eea27f6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.170721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a73d627ff2b4826b3ab025f5eea27f6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.184686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a73d627ff2b4826b3ab025f5eea27f6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.201069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a73d627ff2b4826b3ab025f5eea27f6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.208831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a73d627ff2b4826b3ab025f5eea27f6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.220875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1198/2377] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:38 2022] DELETE /identity/v3/users/9a73d627ff2b4826b3ab025f5eea27f6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:39.506431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:39.524459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2d8f0-ed8b-4626-8595-2609b622a89c service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:39.603710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:39.604361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:39.604704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:39.604990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] PATH_INFO: `/v3/projects/3686c63736c7409eb7cc4170607ca6fb` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:39.606206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3686c63736c7409eb7cc4170607ca6fb)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:39.608117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:39.699074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3686c63736c7409eb7cc4170607ca6fb', 'request_id': 'req-7de643d6-97e8-4734-8672-e301a01136cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.721463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3686c63736c7409eb7cc4170607ca6fb', 'request_id': 'req-7de643d6-97e8-4734-8672-e301a01136cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.749204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3686c63736c7409eb7cc4170607ca6fb', 'request_id': 'req-7de643d6-97e8-4734-8672-e301a01136cb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.780930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] The token cache is being invalidate because project 3686c63736c7409eb7cc4170607ca6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:39.790156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.792993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:39.796670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1180/2378] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:39 2022] DELETE /identity/v3/projects/3686c63736c7409eb7cc4170607ca6fb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:39.801768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:39.803981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:39.833078 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb29627-6ecf-4198-bd09-70c9adad1641 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:39.938379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:39.939142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:39.939824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:39.940444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] PATH_INFO: `/v3/users/83bdbd5980dd4ee8914583bb1a14e1a4` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:39.942069 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bdbd5980dd4ee8914583bb1a14e1a4)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:39.945187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:40.005248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:40.012968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bdbd5980dd4ee8914583bb1a14e1a4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.014201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bdbd5980dd4ee8914583bb1a14e1a4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.023318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bdbd5980dd4ee8914583bb1a14e1a4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.048633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bdbd5980dd4ee8914583bb1a14e1a4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.059501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bdbd5980dd4ee8914583bb1a14e1a4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.067393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1199/2379] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:39 2022] DELETE /identity/v3/users/83bdbd5980dd4ee8914583bb1a14e1a4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:40.371019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:40.372530 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:40.403265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de643d6-97e8-4734-8672-e301a01136cb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:40.494110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:40.494925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:40.495214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:40.495519 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] PATH_INFO: `/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:40.496700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6569ae0dc9ca48e8bb0be3173e5c063c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:40.498688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:40.524684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:40.556725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6569ae0dc9ca48e8bb0be3173e5c063c', 'request_id': 'req-b5d6447f-037c-4778-8419-7af1d8c7d4f2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.563808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6569ae0dc9ca48e8bb0be3173e5c063c', 'request_id': 'req-b5d6447f-037c-4778-8419-7af1d8c7d4f2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.572894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6569ae0dc9ca48e8bb0be3173e5c063c', 'request_id': 'req-b5d6447f-037c-4778-8419-7af1d8c7d4f2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.585709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] The token cache is being invalidate because project 6569ae0dc9ca48e8bb0be3173e5c063c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:40.586033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.586852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b5d6447f-037c-4778-8419-7af1d8c7d4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:40.588421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1181/2380] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:40 2022] DELETE /identity/v3/projects/6569ae0dc9ca48e8bb0be3173e5c063c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:41.764884 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:41.766382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:41.781008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96cc9b-14f5-4706-b37a-7b8e6c661a0a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:41.840922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:41.841684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:41.842119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:41.842472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] PATH_INFO: `/v3/users/95cc4967b70a46318bb73ecc24f20358` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:41.843913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95cc4967b70a46318bb73ecc24f20358)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:41.845899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:41.900854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95cc4967b70a46318bb73ecc24f20358'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:41.901500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95cc4967b70a46318bb73ecc24f20358'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:41.909435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95cc4967b70a46318bb73ecc24f20358'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:41.926514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95cc4967b70a46318bb73ecc24f20358'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:41.934798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95cc4967b70a46318bb73ecc24f20358'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:41.941466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1200/2381] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:41 2022] DELETE /identity/v3/users/95cc4967b70a46318bb73ecc24f20358 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:41.946146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:41.946660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:41.946761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:41.965980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:41.967192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:42.033351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1182/2382] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:42.043370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:42.044815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:42.096044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:42.131680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab347b-1344-4a3e-bd04-578bee6b7fa8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:42.135519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:42.136152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:42.136431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:42.136912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:42.147629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:42.149084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:42.168804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1201/2383] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:42.779423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:42.781045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:42.803937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09faf85-8cc6-45aa-838a-36f34b519061 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:42.810166 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:42.810905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:42.811257 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:42.811612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] PATH_INFO: `/v3/projects/fb9ba1eae2f546cb9da67b8241ea9d1a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:42.812999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb9ba1eae2f546cb9da67b8241ea9d1a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:42.819554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:42.894471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb9ba1eae2f546cb9da67b8241ea9d1a', 'request_id': 'req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:42.902151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb9ba1eae2f546cb9da67b8241ea9d1a', 'request_id': 'req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:42.911451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb9ba1eae2f546cb9da67b8241ea9d1a', 'request_id': 'req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:42.928559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] The token cache is being invalidate because project fb9ba1eae2f546cb9da67b8241ea9d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:42.928918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:42.929917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:42.934639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1183/2384] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:42 2022] DELETE /identity/v3/projects/fb9ba1eae2f546cb9da67b8241ea9d1a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:42.935912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:42.948761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a481-b1ba-4909-a860-77da7d2708b1 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:43.014900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:43.015549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:43.016540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:43.016540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] PATH_INFO: `/v3/users/6e6fdaccd66446f2a4e06ce5701a3912` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:43.017518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6fdaccd66446f2a4e06ce5701a3912)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:43.019777 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:43.086737 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6fdaccd66446f2a4e06ce5701a3912'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.090364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6fdaccd66446f2a4e06ce5701a3912'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.099862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6fdaccd66446f2a4e06ce5701a3912'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.115861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6fdaccd66446f2a4e06ce5701a3912'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.126283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6fdaccd66446f2a4e06ce5701a3912'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.135021 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1202/2385] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:42 2022] DELETE /identity/v3/users/6e6fdaccd66446f2a4e06ce5701a3912 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:43.464052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:43.475536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:43.489162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca140-1a1e-4ba5-a3b8-fe5dffdb1c56 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:43.672601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:43.706390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:43.707052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:43.707387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:43.707731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] PATH_INFO: `/v3/projects/1f2aa4b08fc14682b0ed494236c3a07d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:43.708951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2aa4b08fc14682b0ed494236c3a07d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:43.711041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:43.785745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2aa4b08fc14682b0ed494236c3a07d', 'request_id': 'req-291f3365-4bbd-4a85-a7b9-8bb86157c30e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.792994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2aa4b08fc14682b0ed494236c3a07d', 'request_id': 'req-291f3365-4bbd-4a85-a7b9-8bb86157c30e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.805599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2aa4b08fc14682b0ed494236c3a07d', 'request_id': 'req-291f3365-4bbd-4a85-a7b9-8bb86157c30e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.827238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] The token cache is being invalidate because project 1f2aa4b08fc14682b0ed494236c3a07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:43.827734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.828394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:43.836188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1184/2386] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:43 2022] DELETE /identity/v3/projects/1f2aa4b08fc14682b0ed494236c3a07d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:43.836417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:43.836417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:43.859773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbde4fb-bb4d-4ebb-b7a8-a4e95604c4a9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:43.980292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:43.981055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:43.981355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:43.981682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] PATH_INFO: `/v3/users/fde260f1a83d480887a63483c7a8bf45` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:43.986637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde260f1a83d480887a63483c7a8bf45)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:43.989721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:43.994190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:44.064655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde260f1a83d480887a63483c7a8bf45'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.065376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde260f1a83d480887a63483c7a8bf45'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.075746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde260f1a83d480887a63483c7a8bf45'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.094243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde260f1a83d480887a63483c7a8bf45'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.110177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d5a69ae7-1fee-44a3-9c59-576f44ceac9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde260f1a83d480887a63483c7a8bf45'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.140045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1203/2387] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:43 2022] DELETE /identity/v3/users/fde260f1a83d480887a63483c7a8bf45 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:44.458082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:44.459653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:44.479273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3365-4bbd-4a85-a7b9-8bb86157c30e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:44.564219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:44.564875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:44.565881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:44.566164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] PATH_INFO: `/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:44.567450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6207fd11ef549cc83fbfe5e61d0e36a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:44.569356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:44.623556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6207fd11ef549cc83fbfe5e61d0e36a', 'request_id': 'req-31bb040f-42ed-460f-bf41-3c761029f82b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.630828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6207fd11ef549cc83fbfe5e61d0e36a', 'request_id': 'req-31bb040f-42ed-460f-bf41-3c761029f82b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.643207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6207fd11ef549cc83fbfe5e61d0e36a', 'request_id': 'req-31bb040f-42ed-460f-bf41-3c761029f82b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.648124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:44.657237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] The token cache is being invalidate because project e6207fd11ef549cc83fbfe5e61d0e36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:44.657597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.658525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-31bb040f-42ed-460f-bf41-3c761029f82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:44.660596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1185/2388] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:44 2022] DELETE /identity/v3/projects/e6207fd11ef549cc83fbfe5e61d0e36a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:44.666975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:44.667265 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:44.667544 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:44.689880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:44.691837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:44.750782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1204/2389] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:44.759965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:44.760311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:44.760589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:44.781650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:44.782863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:44.790780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1186/2390] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:44.795203 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:44.806819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb6a49-9bee-4976-a1b3-a7de051c18f3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:44.814559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:44.816624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:44.818056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:44.818587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:44.819435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:56:44.819811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:44.823039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:44.838854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1205/2391] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:56:44.842818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:44.854009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf7fe8-6d44-4446-a304-5d3d9f8051cd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:44.865040 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:44.866115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:44.866494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:44.866889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:44.867734 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:44.869849 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:44.911049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1187/2392] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:44 2022] POST /identity/v3/projects => generated 409 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:44.912711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:44.932817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcf12d-3b3b-4d95-b463-bf49d6950ace admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.005747 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.006627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.007502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.007895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.008746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.011319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.031147 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:45.060609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1206/2393] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:44 2022] POST /identity/v3/users => generated 305 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:45.065990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.077867 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c8cb-357a-426d-8d03-a5f3a35494fb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.084595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.085970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.086516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.087097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.088578 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.091762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.104840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1188/2394] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:45.110004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.133357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c47c-7eb0-4c19-8f55-10d6a072f33e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.138472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.139096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.139504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.139787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.140492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.142134 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efe754-38c9-4c66-be57-3b0ed9fdd5e8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.154083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1207/2395] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:45.158611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.170656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ca22-44d3-472e-a529-8f66f3c9da2a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.176498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.177434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.177965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.178448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] PATH_INFO: `/v3/projects/127d34a9c6e04232b4b76f1c9541e153/users/0dacf38a9df7453ebb0cb62f69a00588/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.201411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127d34a9c6e04232b4b76f1c9541e153, user_id=0dacf38a9df7453ebb0cb62f69a00588, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.204210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.222484 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1189/2396] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:45 2022] PUT /identity/v3/projects/127d34a9c6e04232b4b76f1c9541e153/users/0dacf38a9df7453ebb0cb62f69a00588/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:45.233879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.234170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.234467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.316253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] MFA Rules not processed for user `0dacf38a9df7453ebb0cb62f69a00588`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:45.319290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:45.433205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1208/2397] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:45.437812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.509449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02f0e-fcb2-4e4d-b69c-506a25fa83a2 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.516039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.517086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.517723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.518299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.519196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.521892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.556319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1190/2398] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:45 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:45.561519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.574904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3e556-c704-475b-b4ff-22f0598c628a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.645999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.646803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.647181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.647554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.648685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.650733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.674281 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1209/2399] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:45 2022] POST /identity/v3/users => generated 311 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:45.678821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.690097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab0c1-890e-45c7-a99c-4105b3e1cc7f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.699847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.700906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.701423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.701960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.702955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.704992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.716125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1191/2400] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:45.720439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.732197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceac0c-35f4-4129-84b0-90cacb27b53e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.737766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.738514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.738788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.739072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] PATH_INFO: `/v3/projects/d0ad09b467e149109bd528df101707d5/users/9336160ff2ce4b6b8ed80b8ce59930b6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.763000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad09b467e149109bd528df101707d5, user_id=9336160ff2ce4b6b8ed80b8ce59930b6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.765691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.779712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1210/2401] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:45 2022] PUT /identity/v3/projects/d0ad09b467e149109bd528df101707d5/users/9336160ff2ce4b6b8ed80b8ce59930b6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:56:45.786706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.802564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76df401-4113-4098-8bc8-2f67427f439a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.872617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.873429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.873810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.874210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.875019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.876736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee8c3e-f506-4df7-b1f6-087906561d74 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.885274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:45.892237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1192/2402] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:45.900299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:45.916061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0d86d-c368-496f-8f38-aad90f023018 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:45.923116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:45.923849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:45.924137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:45.924450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] PATH_INFO: `/v3/projects/d0ad09b467e149109bd528df101707d5/users/9336160ff2ce4b6b8ed80b8ce59930b6/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:45.927707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad09b467e149109bd528df101707d5, user_id=9336160ff2ce4b6b8ed80b8ce59930b6, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:45.930780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:45.944702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1211/2403] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:45 2022] PUT /identity/v3/projects/d0ad09b467e149109bd528df101707d5/users/9336160ff2ce4b6b8ed80b8ce59930b6/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:46.022201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:46.022480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:46.022772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:46.051963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] MFA Rules not processed for user `9336160ff2ce4b6b8ed80b8ce59930b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:46.053431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:46.136929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1193/2404] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:46 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:46.148845 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:46.151091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:46.182430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:46.217283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b998b5-e670-4294-8ad3-787e36fa1a54 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:46.222047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:46.224415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:46.224415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:46.224415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:46.239581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:46.241960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:46.258658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1212/2405] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:46.328922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:46.330889 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:46.405245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a1d2-cfa8-4ac1-9e06-18aef75b2712 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:46.409014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:46.409640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:46.409952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:46.410233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:46.412722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:46.447919 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:46.459945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329077d-204c-4c20-a616-9ee08d1bc000 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:46.463860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:46.464542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:46.464918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:46.465268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:46.490080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:46.491690 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:46.502939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1194/2406] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:46.561377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:46.562720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:46.576757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1213/2407] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:56:48.902458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:48.903887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:48.951470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:48.982748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cefcf-9d5b-49df-b8c1-e938177dcb3b service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:48.988066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:48.988675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:48.988951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:48.989221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:48.990969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:49.049969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:49.051553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a04cca-80ed-4088-a9bb-f270045feda3 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:49.061027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1195/2408] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:56:55.164283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:55.252122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59bce-9a67-4334-bab8-1321e3059724 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:55.257651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:55.258451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:55.258760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:55.259056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] PATH_INFO: `/v3/users/0dacf38a9df7453ebb0cb62f69a00588` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:55.260769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dacf38a9df7453ebb0cb62f69a00588)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:55.262851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:55.349600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dacf38a9df7453ebb0cb62f69a00588'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:55.350579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dacf38a9df7453ebb0cb62f69a00588'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:55.361920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dacf38a9df7453ebb0cb62f69a00588'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:55.379675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:55.381209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dacf38a9df7453ebb0cb62f69a00588'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:55.394799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dacf38a9df7453ebb0cb62f69a00588'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:55.405142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1214/2409] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:55 2022] DELETE /identity/v3/users/0dacf38a9df7453ebb0cb62f69a00588 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:55.410791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:55.411998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:55.412649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:55.434125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:55.438773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:55.458159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:55.459604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:55.522814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1196/2410] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:55.532320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:55.536128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2a23-d7a3-47d8-95df-1611ca74a9c0 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:55.539772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:55.540386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:55.540720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:55.541000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:55.542003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39799062-a82d-443b-89ce-add11f2e1bcd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:55.542786 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:55.545995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:55.546631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:55.546972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:55.547220 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:55.554636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:55.556050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efbb97-64a6-4412-a114-07a367f0d919 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:55.556359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:55.557908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:55.566802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1215/2411] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:56:55.568325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1197/2412] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:56:56.053552 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:56.054408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:56.054977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:56.077216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:56.078474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:56.144432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1216/2413] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:56.171010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:56.172761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:56.255935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14793b-fb47-4e3d-be6b-24c9e6805e24 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:56.261521 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:56.262169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:56.262456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:56.262739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:56.268439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=100 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:56.273423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:56.275651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:56.281451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:56.282948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:56.293258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1198/2414] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 12:56:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 12:56:56.317261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dfb02-e9de-42c0-a3ca-151c614c1c98 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:56.324567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:56.325501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:56.325939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:56.330901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] PATH_INFO: `/v3/projects/127d34a9c6e04232b4b76f1c9541e153` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:56.332375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127d34a9c6e04232b4b76f1c9541e153)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:56.335235 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:56.401406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127d34a9c6e04232b4b76f1c9541e153', 'request_id': 'req-f48a2e73-ee58-4d78-b22d-9a3e64de6875'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.409167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127d34a9c6e04232b4b76f1c9541e153', 'request_id': 'req-f48a2e73-ee58-4d78-b22d-9a3e64de6875'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.419114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127d34a9c6e04232b4b76f1c9541e153', 'request_id': 'req-f48a2e73-ee58-4d78-b22d-9a3e64de6875'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.438158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] The token cache is being invalidate because project 127d34a9c6e04232b4b76f1c9541e153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:56.438881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.440923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.445370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1217/2415] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:56 2022] DELETE /identity/v3/projects/127d34a9c6e04232b4b76f1c9541e153 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:56.447274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:56.460548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378d703-5510-4f4f-b462-002cc7913e6e service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:56.536177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:56.537029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:56.537391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:56.537744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] PATH_INFO: `/v3/users/9336160ff2ce4b6b8ed80b8ce59930b6` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:56.539234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9336160ff2ce4b6b8ed80b8ce59930b6)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:56.541459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:56.602046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9336160ff2ce4b6b8ed80b8ce59930b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.610903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9336160ff2ce4b6b8ed80b8ce59930b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.612333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9336160ff2ce4b6b8ed80b8ce59930b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.628424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9336160ff2ce4b6b8ed80b8ce59930b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.629292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-17b3fbc9-b9eb-411a-b93d-2e56d9160dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9336160ff2ce4b6b8ed80b8ce59930b6'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:56.645255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1199/2416] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:56:56 2022] DELETE /identity/v3/users/9336160ff2ce4b6b8ed80b8ce59930b6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:57.085478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.088850 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:57.107617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a2e73-ee58-4d78-b22d-9a3e64de6875 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.147613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:57.208333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.209400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.209944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.210444 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] PATH_INFO: `/v3/projects/d0ad09b467e149109bd528df101707d5` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.211844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ad09b467e149109bd528df101707d5)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.214283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.277034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ad09b467e149109bd528df101707d5', 'request_id': 'req-86795e39-cf1d-43a1-8a44-a0f8136a431f'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:57.284051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ad09b467e149109bd528df101707d5', 'request_id': 'req-86795e39-cf1d-43a1-8a44-a0f8136a431f'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:57.293562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ad09b467e149109bd528df101707d5', 'request_id': 'req-86795e39-cf1d-43a1-8a44-a0f8136a431f'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:57.308017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] The token cache is being invalidate because project d0ad09b467e149109bd528df101707d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:56:57.308341 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:57.309388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-86795e39-cf1d-43a1-8a44-a0f8136a431f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:56:57.311154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1218/2417] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:56:57 2022] DELETE /identity/v3/projects/d0ad09b467e149109bd528df101707d5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:57.319250 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.319789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.320169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.343120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:57.344445 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:57.417680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1200/2418] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:57.431116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.431680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.431959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.452148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:57.453926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:57.462246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1219/2419] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:57.466300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.476914 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb191-56d0-435d-88b4-37436cba7384 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.481691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.482339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.482619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.482925 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.483625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:56:57.483930 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.485533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.496239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:57.500267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1201/2420] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:56:57.505460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.516675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205d3e-c227-4faf-acab-0c4273b1bebe None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.521074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.521694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.522010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.522290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.522957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.524848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.553634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1220/2421] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:57.557795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.568213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158ce3c-3052-4812-8c15-9b0928cc7124 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.628735 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.629352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.629628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.629926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.630625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.632531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.654130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1202/2422] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:56:57.658429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.668972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5cad1-7fc8-4f5d-a6e3-2d54a56c2032 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.673493 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.674648 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.674934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.675201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.675888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.677481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.687558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1221/2423] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:57.692092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.703784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313c335-de42-4187-8973-f01041443e6f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.708601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.709204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.709488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.709762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.710514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.712366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5716769-bd11-41ef-b047-3b0955f85f67 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.724180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1203/2424] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:57.729358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.742857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c51c-7f7b-45df-b417-f8ae2a9d5d27 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.749176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.749890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.750189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.750765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] PATH_INFO: `/v3/projects/b8827c4afe29406d88f7746a5a31e58c/users/5add364df8894402bfd6432890d7f911/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.767807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8827c4afe29406d88f7746a5a31e58c, user_id=5add364df8894402bfd6432890d7f911, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:57.770592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:57.787851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1222/2425] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:57 2022] PUT /identity/v3/projects/b8827c4afe29406d88f7746a5a31e58c/users/5add364df8894402bfd6432890d7f911/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:57.798173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.798638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.799037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.830556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] MFA Rules not processed for user `5add364df8894402bfd6432890d7f911`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:57.831897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:57.909054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1204/2426] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:57.910580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:57.966494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:57.990825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca5aca-fc78-4e77-8a10-a677d0b23874 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:57.996604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:57.997416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:57.997794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:57.998280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:57.999079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.001530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.034074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1223/2427] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:56:57 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:56:58.037997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.049855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b2d9a-1896-4e15-aabf-cac56cef41a3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.121620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.122733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.123231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.123902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.124378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.126623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.144702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:56:58.153966 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1205/2428] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:56:58 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:56:58.156498 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.167666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8cf0-9ff5-4024-8c93-333b98d50e5c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.173097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.173727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.174572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.174904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.176118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.177349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.192704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1224/2429] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:58.193894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.204385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c09e-602c-4916-9bd6-0901458eb148 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.210004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.210669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.210951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.211225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] PATH_INFO: `/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee/users/67c670469df0466ea31ea68647aad454/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.227172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=db55948760ce49fb9dc2d60f75a3e2ee, user_id=67c670469df0466ea31ea68647aad454, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.229850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.245472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1206/2430] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:58 2022] PUT /identity/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee/users/67c670469df0466ea31ea68647aad454/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:58.249338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.260108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef14e2-ab53-448a-9b85-1eb4c56ea6e9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.321926 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.322553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.322838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.323114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.323825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.325443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479d732-165d-4e00-ac5d-64038d447516 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.337492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1225/2431] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:56:58.342049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.355037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd30c2b-b31a-47dc-8320-e303801156bb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.360022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.360789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.361139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.362042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] PATH_INFO: `/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee/users/67c670469df0466ea31ea68647aad454/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.364213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db55948760ce49fb9dc2d60f75a3e2ee, user_id=67c670469df0466ea31ea68647aad454, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.367062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.379339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1207/2432] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:56:58 2022] PUT /identity/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee/users/67c670469df0466ea31ea68647aad454/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:56:58.391126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.391413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.391907 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.429429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] MFA Rules not processed for user `67c670469df0466ea31ea68647aad454`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:56:58.430820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:58.511541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1226/2433] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:56:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:56:58.523156 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:56:58.525008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:56:58.601907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f065a-aaee-4027-b77e-764ad1912c1e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:56:58.605660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:56:58.606322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:56:58.606609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:56:58.606880 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:56:58.664044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:56:58.665482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:56:58.674476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1208/2434] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:56:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:57:02.914222 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:02.916230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:02.971268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:03.013402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d20494-2eed-49de-bfc5-50ba26db2080 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:03.019610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:03.020362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:03.020760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:03.021102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] PATH_INFO: `/v3/users/a1c132b2fdc14e7ca4ffb18109a0fa0d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:03.022443 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c132b2fdc14e7ca4ffb18109a0fa0d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:03.024857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:03.075454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:03.084956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c748e-73d4-435c-82a1-b2173ebb6858 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:03.088865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:03.089486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:03.089773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:03.090146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:03.093456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c132b2fdc14e7ca4ffb18109a0fa0d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.094161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c132b2fdc14e7ca4ffb18109a0fa0d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.102817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:03.103535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c132b2fdc14e7ca4ffb18109a0fa0d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.104786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:03.118666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1209/2435] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:03.122414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c132b2fdc14e7ca4ffb18109a0fa0d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.131811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c132b2fdc14e7ca4ffb18109a0fa0d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.140615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1227/2436] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:02 2022] DELETE /identity/v3/users/a1c132b2fdc14e7ca4ffb18109a0fa0d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:03.335793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:03.349785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986577e4-04a1-4187-b709-d928c3450ac8 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:03.430761 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:03.431488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:03.431843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:03.432297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] PATH_INFO: `/v3/projects/13c13c269e5f46e1a8236c89377c2171` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:03.433790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c13c269e5f46e1a8236c89377c2171)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:03.435706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:03.438439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:03.493939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c13c269e5f46e1a8236c89377c2171', 'request_id': 'req-f571f2a5-441c-43ae-bd86-99b17ea7996d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.500189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c13c269e5f46e1a8236c89377c2171', 'request_id': 'req-f571f2a5-441c-43ae-bd86-99b17ea7996d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.510061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c13c269e5f46e1a8236c89377c2171', 'request_id': 'req-f571f2a5-441c-43ae-bd86-99b17ea7996d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.520546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] The token cache is being invalidate because project 13c13c269e5f46e1a8236c89377c2171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:03.520830 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.521452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.522969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1210/2437] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:03 2022] DELETE /identity/v3/projects/13c13c269e5f46e1a8236c89377c2171 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:03.527071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:03.528474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:03.541934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446907-c000-4df6-be54-130ea893c1a8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:03.602007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:03.602599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:03.602882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:03.603163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] PATH_INFO: `/v3/users/a65cbc8e25e14a9b8a53db8c1cefdd18` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:03.604369 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65cbc8e25e14a9b8a53db8c1cefdd18)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:03.606528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:03.652836 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65cbc8e25e14a9b8a53db8c1cefdd18'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.653483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65cbc8e25e14a9b8a53db8c1cefdd18'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.660783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65cbc8e25e14a9b8a53db8c1cefdd18'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.666240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:03.674474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65cbc8e25e14a9b8a53db8c1cefdd18'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.681610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65cbc8e25e14a9b8a53db8c1cefdd18'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:03.687517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1228/2438] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:03 2022] DELETE /identity/v3/users/a65cbc8e25e14a9b8a53db8c1cefdd18 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:03.867012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:03.868551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:03.883023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571f2a5-441c-43ae-bd86-99b17ea7996d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:03.946524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:03.947171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:03.947452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:03.947724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] PATH_INFO: `/v3/projects/48f444b71c5543b18224dd5174636a53` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:03.948881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48f444b71c5543b18224dd5174636a53)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:03.950833 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:03.997798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48f444b71c5543b18224dd5174636a53', 'request_id': 'req-32800d6b-3a3b-4235-a017-f5deac21bd4d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:04.004333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48f444b71c5543b18224dd5174636a53', 'request_id': 'req-32800d6b-3a3b-4235-a017-f5deac21bd4d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:04.012256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48f444b71c5543b18224dd5174636a53', 'request_id': 'req-32800d6b-3a3b-4235-a017-f5deac21bd4d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:04.023984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] The token cache is being invalidate because project 48f444b71c5543b18224dd5174636a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:04.024297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:04.025406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-32800d6b-3a3b-4235-a017-f5deac21bd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:04.027192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1211/2439] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:03 2022] DELETE /identity/v3/projects/48f444b71c5543b18224dd5174636a53 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:08.151962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:08.153469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:08.169094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5329c-a1bf-4e17-b1ad-a83051dd8157 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:08.231784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:08.231784 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:08.232273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:08.232273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] PATH_INFO: `/v3/users/5add364df8894402bfd6432890d7f911` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:08.233426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5add364df8894402bfd6432890d7f911)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:08.235440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:08.304904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5add364df8894402bfd6432890d7f911'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.309775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5add364df8894402bfd6432890d7f911'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.320682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5add364df8894402bfd6432890d7f911'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.335151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5add364df8894402bfd6432890d7f911'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.342234 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5add364df8894402bfd6432890d7f911'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.349852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1229/2440] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:08 2022] DELETE /identity/v3/users/5add364df8894402bfd6432890d7f911 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:08.354509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:08.355006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:08.355129 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:08.373844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:08.375039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:08.394572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:08.444995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1212/2441] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:08.451721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:08.453214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:08.513875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c182-0c52-451d-be9a-b8ad1e43a193 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:08.518422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:08.519324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:08.519608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:08.519888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:08.529540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:08.531015 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:08.536924 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:08.546232 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1230/2442] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:08.823799 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:08.825326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:08.841520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640b7a5-67c1-485f-a420-34c7f366f023 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:08.848777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:08.849493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:08.850081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:08.850081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] PATH_INFO: `/v3/projects/b8827c4afe29406d88f7746a5a31e58c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:08.851498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8827c4afe29406d88f7746a5a31e58c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:08.853728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:08.915090 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8827c4afe29406d88f7746a5a31e58c', 'request_id': 'req-f1d96005-82c7-4cf7-922e-942b0ab5546d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.921755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8827c4afe29406d88f7746a5a31e58c', 'request_id': 'req-f1d96005-82c7-4cf7-922e-942b0ab5546d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.930582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8827c4afe29406d88f7746a5a31e58c', 'request_id': 'req-f1d96005-82c7-4cf7-922e-942b0ab5546d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.941623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] The token cache is being invalidate because project b8827c4afe29406d88f7746a5a31e58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:08.941954 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.942577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:08.944071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1213/2443] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:08 2022] DELETE /identity/v3/projects/b8827c4afe29406d88f7746a5a31e58c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:08.948085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:08.957676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc5259-d7da-46d7-848d-bd860155428f service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.016436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.017077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.017390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.017655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] PATH_INFO: `/v3/users/67c670469df0466ea31ea68647aad454` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.018842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c670469df0466ea31ea68647aad454)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.020851 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.067754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c670469df0466ea31ea68647aad454'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.068463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c670469df0466ea31ea68647aad454'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.075764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c670469df0466ea31ea68647aad454'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.088199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c670469df0466ea31ea68647aad454'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.095965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-8c5df5ec-a6b7-4c6c-a890-61ff5b25393d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c670469df0466ea31ea68647aad454'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.102518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1231/2444] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:08 2022] DELETE /identity/v3/users/67c670469df0466ea31ea68647aad454 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:09.374804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.376726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:09.391089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96005-82c7-4cf7-922e-942b0ab5546d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.449741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.450325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.450603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.450872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] PATH_INFO: `/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.452050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db55948760ce49fb9dc2d60f75a3e2ee)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.454008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.499167 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:09.500428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db55948760ce49fb9dc2d60f75a3e2ee', 'request_id': 'req-90c1735d-fab4-4fba-a3d0-3890ecce34fd'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.508525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db55948760ce49fb9dc2d60f75a3e2ee', 'request_id': 'req-90c1735d-fab4-4fba-a3d0-3890ecce34fd'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.515559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db55948760ce49fb9dc2d60f75a3e2ee', 'request_id': 'req-90c1735d-fab4-4fba-a3d0-3890ecce34fd'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.526208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] The token cache is being invalidate because project db55948760ce49fb9dc2d60f75a3e2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:09.526501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.527121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-90c1735d-fab4-4fba-a3d0-3890ecce34fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:09.528600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1214/2445] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:09 2022] DELETE /identity/v3/projects/db55948760ce49fb9dc2d60f75a3e2ee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:09.534690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.535175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.535277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.554812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:09.556016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:09.615166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1232/2446] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:09.627560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.628050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.628526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.650118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:09.651338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:09.659652 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1215/2447] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:09.664130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.676471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946859a-f793-43d9-8cd1-19a4668a24d5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.681779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.682475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.682756 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.683027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.683719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:09.684025 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.685628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.697792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1233/2448] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:09.702389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.711549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51f8fc-0032-479e-bf2b-d8efbe47c665 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.716183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.716789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.717069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.717342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.718055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.719960 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.747701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1216/2449] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:09 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:09.752007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.761885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9bc8f-9dfe-4717-acbc-e74441d1c1d7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.765377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:09.820649 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.821350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.821634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.821944 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.822617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.824525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.845268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1234/2450] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:09 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:09.850168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.860345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc8450-91b5-4aff-9b42-4cf29e2a76db admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.865054 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.865656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.865981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.866258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.866953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.868536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.878884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1217/2451] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:09.882605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.892960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7beeab-2324-4436-83de-c873ed4bf93c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.897694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.898343 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.898624 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.898903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.900670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.901186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08f1d-74fd-4551-8f95-c21dfdc940e8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.912619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1235/2452] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:09.916819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:09.929597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf2a98-16a7-491f-b0cf-bb43b915324c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:09.934292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:09.934953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.935272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.935509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] PATH_INFO: `/v3/projects/7b1add18ecf94c3eb1078aaa5866a8a0/users/cfe50161271849988351391d094849d0/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:09.958081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1add18ecf94c3eb1078aaa5866a8a0, user_id=cfe50161271849988351391d094849d0, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:09.962912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:09.978838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1218/2453] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:09 2022] PUT /identity/v3/projects/7b1add18ecf94c3eb1078aaa5866a8a0/users/cfe50161271849988351391d094849d0/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:09.992240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:09.992600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:09.992958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:10.023093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] MFA Rules not processed for user `cfe50161271849988351391d094849d0`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:10.024892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:10.097510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1236/2454] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:10.109586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:10.111459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:10.168315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848a184-bade-4be2-9868-adf585100103 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:10.171926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:10.172535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:10.172814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:10.173083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:10.181488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:10.182891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:10.191872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1219/2455] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:12.480229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:12.481651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:12.539581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e107fe-4fa9-4ec1-8c08-c62b37238961 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:12.543912 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:12.544532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:12.544814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:12.545089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:12.546854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:12.585960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:12.622161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:12.623608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:12.633955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1237/2456] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:15.579025 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:15.581209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:15.660959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:15.662413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:15.670310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c610f-ed84-4987-a125-c327c9be07d9 service glance] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:15.677996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e7fe0-48ee-4605-b501-e2e60d698bb6 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:15.679132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:15.681536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:15.681870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:15.682205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:15.683034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:15.683720 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:15.684000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:15.684157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:15.684266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] PATH_INFO: `/v3/users/b4d57a4e83ab4310b5b3ba90f43c5ddc` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:15.685438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d57a4e83ab4310b5b3ba90f43c5ddc)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:15.687397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:15.701873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:15.703498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:15.716070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1220/2457] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:15.743506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d57a4e83ab4310b5b3ba90f43c5ddc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:15.744162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d57a4e83ab4310b5b3ba90f43c5ddc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:15.753244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d57a4e83ab4310b5b3ba90f43c5ddc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:15.766075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d57a4e83ab4310b5b3ba90f43c5ddc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:15.773008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-502bf815-219d-4cc9-8d5c-efd8fdd01831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d57a4e83ab4310b5b3ba90f43c5ddc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:15.779059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1238/2458] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:15 2022] DELETE /identity/v3/users/b4d57a4e83ab4310b5b3ba90f43c5ddc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:15.966105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:15.975228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:15.980732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083c64-a4f8-4267-b060-35d94b5fe84f service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.044464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.045228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.045586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.045888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] PATH_INFO: `/v3/projects/d57fc58eac4149e780e4329e87582260` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.047034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57fc58eac4149e780e4329e87582260)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.049453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.097476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57fc58eac4149e780e4329e87582260', 'request_id': 'req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:16.103456 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57fc58eac4149e780e4329e87582260', 'request_id': 'req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:16.113287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57fc58eac4149e780e4329e87582260', 'request_id': 'req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:16.124596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] The token cache is being invalidate because project d57fc58eac4149e780e4329e87582260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:16.124884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:16.125549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abfd99bf-6a82-4f62-86d6-39fb1ca3f67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:16.127772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1221/2459] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:15 2022] DELETE /identity/v3/projects/d57fc58eac4149e780e4329e87582260 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:16.132622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.132988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.133339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.153042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:16.154266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:16.214705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1239/2460] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:16.222864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54531-acdb-469b-975d-def05d82855b None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.223193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54531-acdb-469b-975d-def05d82855b None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.223544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54531-acdb-469b-975d-def05d82855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.244586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-25c54531-acdb-469b-975d-def05d82855b None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:16.245854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-25c54531-acdb-469b-975d-def05d82855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:16.254038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1222/2461] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:16.258447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.268667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121894a9-58ed-463d-a6e0-558f5f216405 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.273388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.274104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.274392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.274668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.275365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:16.275673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.277267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.289533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1240/2462] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:16.294253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54531-acdb-469b-975d-def05d82855b None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.304993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54531-acdb-469b-975d-def05d82855b None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.309870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.310489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.310773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.311076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.311744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.313651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.341553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1223/2463] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:16.345805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.356744 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496c297-275c-4449-a844-130960ce0e12 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.379761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:16.416942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.417562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.417878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.418158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.418831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.420774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.442805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1241/2464] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:16.446986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.458736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8dc-1a74-414b-bdc9-b7a70b6b133e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.463487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.464098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.464375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.464647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.465340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.467075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.482116 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1224/2465] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:16.483082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.498198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8dcb-7baa-42b2-86f6-8d3cb3d05c5b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.504197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.504843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.505127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.505414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.506315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.508008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de1f6f-850f-4db3-9361-82b4d471f096 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.519671 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1242/2466] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:57:16.523785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.534272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d783-fa01-4494-96d4-3e5ab04d9e72 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.539085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.539806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.540086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.540363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] PATH_INFO: `/v3/projects/0e8260e58d0f4955bd920eaa665451c4/users/01049cb77845409ba039ec540c5df4cf/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.558462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8260e58d0f4955bd920eaa665451c4, user_id=01049cb77845409ba039ec540c5df4cf, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.561415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.576573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1225/2467] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:16 2022] PUT /identity/v3/projects/0e8260e58d0f4955bd920eaa665451c4/users/01049cb77845409ba039ec540c5df4cf/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:16.590665 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.591991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.593921 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.625429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] MFA Rules not processed for user `01049cb77845409ba039ec540c5df4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:16.626643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:16.690916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1243/2468] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:57:16.695561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.729619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:16.761608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348a76-b688-4b37-b07c-92dfc7fbbb53 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.766347 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.766967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.767804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.768082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.768751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.770683 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.796110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1226/2469] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:16.800236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.811258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23c69d-19f8-49b6-9b73-854fea2edb4e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.870488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.871110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.871389 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.871660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.872331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.874293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.896215 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1244/2470] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:16 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:16.900251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.913623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1737-cdbc-44bb-aa6c-4ad3cfc9cb74 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.917021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.917716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.918052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.918675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.919390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.921403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:16.937081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1227/2471] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:16.942047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:16.952559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92eec-f15e-4be1-bffd-e4fbd10ccee9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:16.958003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:16.958730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:16.958999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:16.959282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] PATH_INFO: `/v3/projects/14605ba0381e44e59cfc266272a2bbc8/users/e5183c3d943741b1b65e2b87bdf1da7b/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:16.993731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] RBAC: Authorizing `identity:create_grant(project_id=14605ba0381e44e59cfc266272a2bbc8, user_id=e5183c3d943741b1b65e2b87bdf1da7b, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:16.996471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:17.005696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:17.012796 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1245/2472] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:16 2022] PUT /identity/v3/projects/14605ba0381e44e59cfc266272a2bbc8/users/e5183c3d943741b1b65e2b87bdf1da7b/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:17.017435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:17.027580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222fc0-37b1-4b39-ab99-abace9d898b0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:17.083854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:17.084471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.084751 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.085022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.085713 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:17.087353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a648b-440f-4464-9c20-45966c8a4f24 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:17.097296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1228/2473] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:17.101492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:17.112637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424b0a7-eb88-48b1-9e98-80e41ddb21da admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:17.117268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:17.117962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.118240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.118514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] PATH_INFO: `/v3/projects/14605ba0381e44e59cfc266272a2bbc8/users/e5183c3d943741b1b65e2b87bdf1da7b/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.120442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14605ba0381e44e59cfc266272a2bbc8, user_id=e5183c3d943741b1b65e2b87bdf1da7b, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:17.123081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:17.133440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1246/2474] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:17 2022] PUT /identity/v3/projects/14605ba0381e44e59cfc266272a2bbc8/users/e5183c3d943741b1b65e2b87bdf1da7b/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:17.145587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.145908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.146184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.173813 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] MFA Rules not processed for user `e5183c3d943741b1b65e2b87bdf1da7b`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:17.174986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:17.244776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1229/2475] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:17.256097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:17.257793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:17.332300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c154c-984d-4043-ac5a-61c465208880 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:17.336086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:17.336732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.337019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.337300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.348194 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:17.349633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3db0f-51dc-408c-ba03-408f74ed5d73 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:17.359014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1247/2476] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:17.783438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:17.814874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:17.848906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454061cf-6319-4710-bf9d-ab6555e975f0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:17.853308 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:17.853982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.854264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.854531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] PATH_INFO: `/v3/users/01049cb77845409ba039ec540c5df4cf` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.855676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01049cb77845409ba039ec540c5df4cf)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:17.857603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:17.903191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01049cb77845409ba039ec540c5df4cf'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:17.910617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01049cb77845409ba039ec540c5df4cf'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:17.911289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01049cb77845409ba039ec540c5df4cf'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:17.929924 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01049cb77845409ba039ec540c5df4cf'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:17.930756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01049cb77845409ba039ec540c5df4cf'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:17.943522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1230/2477] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:17 2022] DELETE /identity/v3/users/01049cb77845409ba039ec540c5df4cf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:17.948166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:17.948646 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:17.948774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:17.967579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:17.968753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:18.039370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1248/2478] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:18.049135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:18.050704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:18.110021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973ab1-69a0-4731-aee8-3824f0e2873c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:18.113844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:18.114431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:18.114712 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:18.114983 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:18.124407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:18.125855 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:18.135798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1231/2479] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:18.416729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:18.418512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:18.432982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c01d2-5499-4d22-ab3c-61af6aa54edb None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:18.438142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:18.438960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:18.439308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:18.439606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] PATH_INFO: `/v3/projects/0e8260e58d0f4955bd920eaa665451c4` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:18.440795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8260e58d0f4955bd920eaa665451c4)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:18.443597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:18.491803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8260e58d0f4955bd920eaa665451c4', 'request_id': 'req-2e2e1c03-ac78-4d8b-8760-ff91d1961603'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.497769 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8260e58d0f4955bd920eaa665451c4', 'request_id': 'req-2e2e1c03-ac78-4d8b-8760-ff91d1961603'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.506359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8260e58d0f4955bd920eaa665451c4', 'request_id': 'req-2e2e1c03-ac78-4d8b-8760-ff91d1961603'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.518254 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] The token cache is being invalidate because project 0e8260e58d0f4955bd920eaa665451c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:18.518549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.519178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.520696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1249/2480] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:18 2022] DELETE /identity/v3/projects/0e8260e58d0f4955bd920eaa665451c4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:18.524858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:18.535783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19a6a-3f9f-4d4e-b02a-c15dafc48407 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:18.583681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:18.598756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:18.599389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:18.599665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:18.599936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] PATH_INFO: `/v3/users/e5183c3d943741b1b65e2b87bdf1da7b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:18.601074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5183c3d943741b1b65e2b87bdf1da7b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:18.603001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:18.656432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5183c3d943741b1b65e2b87bdf1da7b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.663885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5183c3d943741b1b65e2b87bdf1da7b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.664544 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5183c3d943741b1b65e2b87bdf1da7b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.676209 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5183c3d943741b1b65e2b87bdf1da7b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.676838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-edb94c7c-ffe4-4a80-9fdd-a49084847217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5183c3d943741b1b65e2b87bdf1da7b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:18.689749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1232/2481] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:18 2022] DELETE /identity/v3/users/e5183c3d943741b1b65e2b87bdf1da7b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:18.970754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:18.972472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:18.985453 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e1c03-ac78-4d8b-8760-ff91d1961603 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:18.994218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:19.050026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.050690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.051001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.051278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] PATH_INFO: `/v3/projects/14605ba0381e44e59cfc266272a2bbc8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.052591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14605ba0381e44e59cfc266272a2bbc8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.054551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.104827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14605ba0381e44e59cfc266272a2bbc8', 'request_id': 'req-df068610-f336-4612-9f8d-c74600fe3332'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:19.110463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14605ba0381e44e59cfc266272a2bbc8', 'request_id': 'req-df068610-f336-4612-9f8d-c74600fe3332'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:19.119606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14605ba0381e44e59cfc266272a2bbc8', 'request_id': 'req-df068610-f336-4612-9f8d-c74600fe3332'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:19.131883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] The token cache is being invalidate because project 14605ba0381e44e59cfc266272a2bbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:19.132175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:19.132824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-df068610-f336-4612-9f8d-c74600fe3332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:19.134362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1250/2482] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:18 2022] DELETE /identity/v3/projects/14605ba0381e44e59cfc266272a2bbc8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:19.139283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.139770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.139886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.160093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:19.161279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:19.220887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1233/2483] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:19.228755 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.229046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.229327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.252492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:19.253713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:19.261573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1251/2484] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:19.265892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.276225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246740cf-e4f1-4d64-95f8-609e6103e52c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.280785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.281397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.281728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.282049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.283404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:19.283711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.285304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.296744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1234/2485] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:19.300738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.310032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4893cf-b949-4871-91e1-2317c9d81d4e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.314900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.315638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.315937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.316197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.316880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.318889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.347662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1252/2486] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:19.351818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.361843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4c468-ba71-4cca-89cf-ed23ecce2fc6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.411888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:19.419089 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.420061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.420359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.420619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.421284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.423269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.444724 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1235/2487] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:19.448838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.460200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd3b3d-debf-4716-9a88-279549290ae9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.464840 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.465494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.465772 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.466110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.466835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.468418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.479206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1253/2488] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:19.483330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.493702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0089bbc-a9f6-4f95-b720-c14b00528a7e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.498386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.499039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.499342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.499614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.500331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.502543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81803-d7a6-48ac-ba70-4369f0359ba1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.514195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1236/2489] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:19.518993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.528763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8842-9b3d-42cb-b123-8a7a45d00468 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.533644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.534328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.534609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.534880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] PATH_INFO: `/v3/projects/c99417ab181341a39fe83d3fb1557fbf/users/6164f3b2ab2945529859ee2f1724a3fb/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.549696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99417ab181341a39fe83d3fb1557fbf, user_id=6164f3b2ab2945529859ee2f1724a3fb, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.552284 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.566588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1254/2490] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:19 2022] PUT /identity/v3/projects/c99417ab181341a39fe83d3fb1557fbf/users/6164f3b2ab2945529859ee2f1724a3fb/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:19.577426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.577706 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.578014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.605670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] MFA Rules not processed for user `6164f3b2ab2945529859ee2f1724a3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:19.606907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:19.676694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1237/2491] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:19.681135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.726265 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:19.749067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae85-9e8d-4dd0-a167-e934983d43ed admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.754342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.754960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.755247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.755525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.756216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.758167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.786548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1255/2492] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:19.790774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.801373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a5e32-8115-4aec-bb3e-5a8b7cf1ba58 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.858997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.859621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.859905 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.860181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.861390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.863509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.884465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1238/2493] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:19 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:19.888770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.900393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187edbd0-635c-46d7-bb8c-45e81d349363 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.905346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.906121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.906413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.906691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.907400 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.909622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.920928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1256/2494] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:19.925434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.935617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e743d8-c4bd-4f36-a145-ad6818eb852f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:19.940285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:19.940937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:19.941218 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:19.941490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] PATH_INFO: `/v3/projects/cf66a07b56804ba6a0cdf68ce31af578/users/f0d10acf6b74487aa9f09921fdd3c658/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:19.961046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf66a07b56804ba6a0cdf68ce31af578, user_id=f0d10acf6b74487aa9f09921fdd3c658, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:19.963715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:19.977780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1239/2495] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:19 2022] PUT /identity/v3/projects/cf66a07b56804ba6a0cdf68ce31af578/users/f0d10acf6b74487aa9f09921fdd3c658/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:19.981804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:19.992296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a939-5705-42d4-ac11-d84097750a36 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.049893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.050518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.050799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.051075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.051821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.053346 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4440fab-6ddb-4f22-83cc-37bb27d23b78 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.063027 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1257/2496] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:20.068110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.078819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511e13d-1b26-4f10-88c3-8c03ddb6b22b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.083390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.084052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.084332 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.084606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] PATH_INFO: `/v3/projects/cf66a07b56804ba6a0cdf68ce31af578/users/f0d10acf6b74487aa9f09921fdd3c658/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.086552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf66a07b56804ba6a0cdf68ce31af578, user_id=f0d10acf6b74487aa9f09921fdd3c658, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.089181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.099873 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1240/2497] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:20 2022] PUT /identity/v3/projects/cf66a07b56804ba6a0cdf68ce31af578/users/f0d10acf6b74487aa9f09921fdd3c658/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:20.111904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.112179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.112454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.145318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] MFA Rules not processed for user `f0d10acf6b74487aa9f09921fdd3c658`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:20.146582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:20.214598 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1258/2498] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:20.226907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.228621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:20.247455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:20.289213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c7c8-96af-4ad3-9bb9-1ef2add911f7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.292774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.293387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.293665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.293976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.303109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.304501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.314806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1241/2499] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:20.627899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.637699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03248fb-1ec6-4919-b68e-18d919a63caf None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.642599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d10acf6b74487aa9f09921fdd3c658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf66a07b56804ba6a0cdf68ce31af578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.643219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.643513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.643794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.644530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:20.644839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.646480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.661442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1259/2500] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:57:20.664913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.674467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81b601-a48e-4167-b589-9271c4621213 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.678977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d10acf6b74487aa9f09921fdd3c658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf66a07b56804ba6a0cdf68ce31af578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.679581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.679864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.680143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.680816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.682797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.709371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1242/2501] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:20 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:20.713459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.724275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c942b7c-914a-4c26-ba26-175521fdd266 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.742632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:20.783816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d10acf6b74487aa9f09921fdd3c658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf66a07b56804ba6a0cdf68ce31af578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.784440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.784725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.785008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.785703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:20.786048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.787664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.799506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1260/2502] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:57:20.836762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:20.847571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f74d-5438-403c-b51c-6820c65f1d6d tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:20.852101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d10acf6b74487aa9f09921fdd3c658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf66a07b56804ba6a0cdf68ce31af578', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:20.852727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:20.853011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:20.853289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] PATH_INFO: `/v3/projects/b255204f45534d8fb221af6ada4b33da` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:20.854483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b255204f45534d8fb221af6ada4b33da)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:20.856370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:20.905249 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b255204f45534d8fb221af6ada4b33da', 'request_id': 'req-b03a1c9f-31d9-431f-80ba-7c84bd91c249'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:20.912013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b255204f45534d8fb221af6ada4b33da', 'request_id': 'req-b03a1c9f-31d9-431f-80ba-7c84bd91c249'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:20.919859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b255204f45534d8fb221af6ada4b33da', 'request_id': 'req-b03a1c9f-31d9-431f-80ba-7c84bd91c249'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:20.931107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] The token cache is being invalidate because project b255204f45534d8fb221af6ada4b33da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:20.931397 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:20.932041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b03a1c9f-31d9-431f-80ba-7c84bd91c249 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:20.933562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1243/2503] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:20 2022] DELETE /identity/v3/projects/b255204f45534d8fb221af6ada4b33da => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:21.131846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:21.133549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:21.201799 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f35336-5ecf-4d2c-9047-140f8bfdb052 tempest-VolumeQuotaClassesTest-192756055 tempest-VolumeQuotaClassesTest-192756055-project-admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:21.258985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:21.259631 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:21.259913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:21.260190 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] PATH_INFO: `/v3/users/6164f3b2ab2945529859ee2f1724a3fb` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:21.261355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6164f3b2ab2945529859ee2f1724a3fb)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:21.263367 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:21.312726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6164f3b2ab2945529859ee2f1724a3fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.313382 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6164f3b2ab2945529859ee2f1724a3fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.321995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6164f3b2ab2945529859ee2f1724a3fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.329645 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:21.335523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6164f3b2ab2945529859ee2f1724a3fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.343334 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6164f3b2ab2945529859ee2f1724a3fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.349622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1261/2504] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:21 2022] DELETE /identity/v3/users/6164f3b2ab2945529859ee2f1724a3fb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:21.354087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:21.354570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:21.354680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:21.373396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:21.374562 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:21.432585 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1244/2505] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:21.443732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:21.445191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:21.503872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab54189-8c91-4862-b68f-ee05b27d00b9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:21.507748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:21.508435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:21.508723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:21.508999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:21.518558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:21.520039 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:21.529588 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1262/2506] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:21.799739 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:21.801247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:21.813549 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df272f34-007c-46ac-836b-ff6750328c38 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:21.823253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:21.823910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:21.824188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:21.824461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] PATH_INFO: `/v3/projects/c99417ab181341a39fe83d3fb1557fbf` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:21.825621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99417ab181341a39fe83d3fb1557fbf)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:21.827554 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:21.844753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:21.876780 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99417ab181341a39fe83d3fb1557fbf', 'request_id': 'req-b551934f-0b86-4282-9eaf-9a14c721347c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.882820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99417ab181341a39fe83d3fb1557fbf', 'request_id': 'req-b551934f-0b86-4282-9eaf-9a14c721347c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.889749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99417ab181341a39fe83d3fb1557fbf', 'request_id': 'req-b551934f-0b86-4282-9eaf-9a14c721347c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.900517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] The token cache is being invalidate because project c99417ab181341a39fe83d3fb1557fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:21.900820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.901429 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:21.902944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1245/2507] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:21 2022] DELETE /identity/v3/projects/c99417ab181341a39fe83d3fb1557fbf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:21.906909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:21.917074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38f566-c2b5-4214-88bc-3c615b20313d service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:21.973971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:21.974609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:21.974893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:21.975168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] PATH_INFO: `/v3/users/f0d10acf6b74487aa9f09921fdd3c658` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:21.976434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d10acf6b74487aa9f09921fdd3c658)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:21.978590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.024770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d10acf6b74487aa9f09921fdd3c658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.025419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d10acf6b74487aa9f09921fdd3c658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.033034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d10acf6b74487aa9f09921fdd3c658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.046226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d10acf6b74487aa9f09921fdd3c658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.053591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9c7eb2dd-f95e-40b3-88aa-da411d78f4d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d10acf6b74487aa9f09921fdd3c658'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.059558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1263/2508] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:21 2022] DELETE /identity/v3/users/f0d10acf6b74487aa9f09921fdd3c658 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:22.352182 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.353882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:22.369606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551934f-0b86-4282-9eaf-9a14c721347c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.426845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.427444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.427726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.428002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] PATH_INFO: `/v3/projects/cf66a07b56804ba6a0cdf68ce31af578` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.429157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf66a07b56804ba6a0cdf68ce31af578)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.431093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.476728 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf66a07b56804ba6a0cdf68ce31af578', 'request_id': 'req-fa3da7b3-794f-495a-bd13-67dd69c3ab21'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.482717 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf66a07b56804ba6a0cdf68ce31af578', 'request_id': 'req-fa3da7b3-794f-495a-bd13-67dd69c3ab21'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.490814 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf66a07b56804ba6a0cdf68ce31af578', 'request_id': 'req-fa3da7b3-794f-495a-bd13-67dd69c3ab21'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.501233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] The token cache is being invalidate because project cf66a07b56804ba6a0cdf68ce31af578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:22.501526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.502175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fa3da7b3-794f-495a-bd13-67dd69c3ab21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:22.503669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1246/2509] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:22 2022] DELETE /identity/v3/projects/cf66a07b56804ba6a0cdf68ce31af578 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:22.509287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.509760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.509875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.517470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:22.530824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:22.532034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:22.590433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1264/2510] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:22.598390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.598678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.598951 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.619145 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:22.620480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:22.629111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1247/2511] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:22.633120 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.642898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6da10a-60e5-4e07-9363-66f0c2768d68 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.647511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.648121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.648406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.648687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.649409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:22.649716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.651383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.662891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1265/2512] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:22.667237 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.676927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b7b8d-dcd5-475e-b095-db79b6b57053 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.681708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.682322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.682601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.682871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.683555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.685424 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.713120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1248/2513] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:22 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:22.717264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.727835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111f2-7d6f-4a73-959c-690481c9a54d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.786030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.786656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.787023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.787304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.787975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.789940 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.811715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1266/2514] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:22 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:22.816598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.823274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:22.827301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1145d90-6895-4300-9f25-1951013c131c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.832912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.833804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.834325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.834816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.835553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.837139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.847243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1249/2515] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:22.851012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.861406 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5eb11-3790-4edb-8b2e-74db88011f80 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.866081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.866689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.866974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.867256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.867953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.869539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1b5b7-f388-44d8-bc6e-7067f6dec6fe admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.880164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1267/2516] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:22.884175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:22.894678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85bcb-a16d-481e-b312-d7db2a3da2cf admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:22.899451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:22.900108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.900389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.900662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] PATH_INFO: `/v3/projects/0f1f2f25c8be407ab61913287606a332/users/61cfac749e5a476080a8b94390e558b3/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.916955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1f2f25c8be407ab61913287606a332, user_id=61cfac749e5a476080a8b94390e558b3, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:22.920297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:22.934368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1250/2517] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:22 2022] PUT /identity/v3/projects/0f1f2f25c8be407ab61913287606a332/users/61cfac749e5a476080a8b94390e558b3/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:22.946294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:22.946577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:22.946878 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:22.974137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] MFA Rules not processed for user `61cfac749e5a476080a8b94390e558b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:22.975321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:23.033323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:23.039974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1268/2518] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:23.044650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.109796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c75d6-507a-47dd-9d4e-2f4fc7d85a36 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.114901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.115526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.115807 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.116083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.116946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.118892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.149141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1251/2519] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:23 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:23.153430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.164274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248a6ee-27e8-4b46-85e8-56da944388ed None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.222947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.223517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.223802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.224082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.224758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.226721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.247893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1269/2520] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:23 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:23.251926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.262606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066b142-9869-4da3-b167-24a60a49a006 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.267315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.267921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.268200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.268467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.269153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.270764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.281406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1252/2521] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:23.286726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.299415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e242abe-ba38-41c7-9a41-69c18b311140 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.304297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.304999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.305277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.305637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] PATH_INFO: `/v3/projects/10eddd9d33f14934a3702d3ed636ce8b/users/bf3a71dbe367415daf5646051fd2c54a/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.321692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10eddd9d33f14934a3702d3ed636ce8b, user_id=bf3a71dbe367415daf5646051fd2c54a, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.324423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.336509 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1270/2522] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:23 2022] PUT /identity/v3/projects/10eddd9d33f14934a3702d3ed636ce8b/users/bf3a71dbe367415daf5646051fd2c54a/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:23.340466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.351248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39a905-1674-4720-bfce-c649140015d7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.412987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.413662 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.413891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.414163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.414843 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.416444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6371e-9d94-462c-a1fa-a220b36be12e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.426650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1253/2523] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:23.430825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.442174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8b89-1ae0-463e-9b18-46f99ed0f1b9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.447592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.448253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.448536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.448813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] PATH_INFO: `/v3/projects/10eddd9d33f14934a3702d3ed636ce8b/users/bf3a71dbe367415daf5646051fd2c54a/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.450913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10eddd9d33f14934a3702d3ed636ce8b, user_id=bf3a71dbe367415daf5646051fd2c54a, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.453706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.465006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1271/2524] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:23 2022] PUT /identity/v3/projects/10eddd9d33f14934a3702d3ed636ce8b/users/bf3a71dbe367415daf5646051fd2c54a/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:23.475555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.475787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.476069 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.503854 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] MFA Rules not processed for user `bf3a71dbe367415daf5646051fd2c54a`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:23.505046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:23.534866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:23.574525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1254/2525] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:23.585673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.587468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:23.644123 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaa5e3-6653-4a7b-a2be-7aea4e104142 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.647783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.648394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.648677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.648956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.659524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.660933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccdbe6-7370-497e-aad5-891300817ba5 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.670247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1272/2526] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:23.805458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:23.867455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a136b-3717-4e1d-8f70-c3d9069380c1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:23.871977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:23.872601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.872877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.873147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] PATH_INFO: `/v3/users/61cfac749e5a476080a8b94390e558b3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.874302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61cfac749e5a476080a8b94390e558b3)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:23.876198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:23.924161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61cfac749e5a476080a8b94390e558b3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:23.931213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61cfac749e5a476080a8b94390e558b3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:23.931870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61cfac749e5a476080a8b94390e558b3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:23.944168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61cfac749e5a476080a8b94390e558b3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:23.944794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61cfac749e5a476080a8b94390e558b3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:23.958462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1255/2527] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:23 2022] DELETE /identity/v3/users/61cfac749e5a476080a8b94390e558b3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:23.963062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:23.963375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:23.963655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:23.982263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:23.983392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:24.028539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:24.043503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1273/2528] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:24.053316 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:24.054805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:24.104608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:24.118086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d106f0-93cc-4147-b132-e7a5b3182700 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:24.122174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:24.122797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:24.123096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:24.123342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:24.134412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:24.135845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:24.145407 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1256/2529] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:24.419129 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:24.420718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:24.433948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745e0b5-a4ac-4796-8c63-b6998ec37b12 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:24.439694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:24.440262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:24.440551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:24.440837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] PATH_INFO: `/v3/projects/0f1f2f25c8be407ab61913287606a332` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:24.442165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1f2f25c8be407ab61913287606a332)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:24.444105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:24.491405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1f2f25c8be407ab61913287606a332', 'request_id': 'req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.496686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1f2f25c8be407ab61913287606a332', 'request_id': 'req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.504131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1f2f25c8be407ab61913287606a332', 'request_id': 'req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.515128 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] The token cache is being invalidate because project 0f1f2f25c8be407ab61913287606a332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:24.515416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.516038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.517533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1274/2530] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:24 2022] DELETE /identity/v3/projects/0f1f2f25c8be407ab61913287606a332 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:24.521470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:24.531380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965b953-0be9-4f52-994e-f90b974533ba service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:24.586582 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:24.587217 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:24.587498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:24.587771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] PATH_INFO: `/v3/users/bf3a71dbe367415daf5646051fd2c54a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:24.588917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3a71dbe367415daf5646051fd2c54a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:24.590868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:24.636779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3a71dbe367415daf5646051fd2c54a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.643653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3a71dbe367415daf5646051fd2c54a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.644358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3a71dbe367415daf5646051fd2c54a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.656359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3a71dbe367415daf5646051fd2c54a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.656996 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b8ab8703-2a8c-4614-8510-cd225bea50c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3a71dbe367415daf5646051fd2c54a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:24.669573 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1257/2531] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:24 2022] DELETE /identity/v3/users/bf3a71dbe367415daf5646051fd2c54a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:24.957307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:24.959295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:24.973372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed439-2ae2-4c6a-a8a6-d457e1354ba1 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.029917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.030496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.030783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.031063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] PATH_INFO: `/v3/projects/10eddd9d33f14934a3702d3ed636ce8b` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.032245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10eddd9d33f14934a3702d3ed636ce8b)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.035023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.080436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10eddd9d33f14934a3702d3ed636ce8b', 'request_id': 'req-69f57352-4557-41cd-aed1-d0c3a7a28c6e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:25.085950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10eddd9d33f14934a3702d3ed636ce8b', 'request_id': 'req-69f57352-4557-41cd-aed1-d0c3a7a28c6e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:25.092964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10eddd9d33f14934a3702d3ed636ce8b', 'request_id': 'req-69f57352-4557-41cd-aed1-d0c3a7a28c6e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:25.105058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] The token cache is being invalidate because project 10eddd9d33f14934a3702d3ed636ce8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:25.105347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:25.106019 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-69f57352-4557-41cd-aed1-d0c3a7a28c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:25.107520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1275/2532] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:24 2022] DELETE /identity/v3/projects/10eddd9d33f14934a3702d3ed636ce8b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:25.112426 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.112716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.112990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.135198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:25.136234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:25.201336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1258/2533] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:25.209672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.209985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.210267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.226257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:25.240374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:25.241618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:25.249876 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1276/2534] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:25.254314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.264365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10353cd-39d3-43f2-a6e0-3e524a8c50e6 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.269235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.269903 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.270242 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.270546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.271240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:25.271546 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.273274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.274777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:25.287386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1259/2535] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:25.291573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.301431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df011-dae8-4fa5-8fb2-725b641ec4e1 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.306213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.306821 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.307107 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.307411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.308195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.310198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.344085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1277/2536] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:25 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:25.349010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.359327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb530948-e13a-492c-8d73-9078e177ece0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.419981 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.420604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.420956 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.421244 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.421958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.424550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.445050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1260/2537] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:25 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:25.449227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.462428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90c05-ed78-4f82-b90f-7a765780e3d9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.467248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.467923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.468211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.468490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.469480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.472096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.487963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1278/2538] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:25.491927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.502368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18200-39d1-44bb-a61d-923d994bbdc2 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.507617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.508232 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.508508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.508784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.509485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.511122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a6472-3228-4c5c-8fa6-22bce7538c38 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.522199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1261/2539] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:25.526536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.538306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea268-3537-46d7-b398-cd0fb0b5194d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.542918 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.543583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.543862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.544135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] PATH_INFO: `/v3/projects/20fcc3e0a1f343e8835ccc9201668267/users/4d7231134519451c966eca1f25e52794/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.561161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fcc3e0a1f343e8835ccc9201668267, user_id=4d7231134519451c966eca1f25e52794, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.563937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.578035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1279/2540] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:25 2022] PUT /identity/v3/projects/20fcc3e0a1f343e8835ccc9201668267/users/4d7231134519451c966eca1f25e52794/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:25.583748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9774-5ecb-4c25-becf-eafb1de6ddf1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.584036 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9774-5ecb-4c25-becf-eafb1de6ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.584310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9774-5ecb-4c25-becf-eafb1de6ddf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.603969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-967a9774-5ecb-4c25-becf-eafb1de6ddf1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:25.605184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-967a9774-5ecb-4c25-becf-eafb1de6ddf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:25.663349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1262/2541] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:25.672999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:25.674757 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:25.737808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e3ee2-c03a-4a83-ab00-945a57e65a04 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:25.741719 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:25.742687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:25.743188 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:25.743670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:25.754392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:25.755843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:25.765445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1280/2542] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:26.681527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:26.682064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:26.682256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:26.714550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] MFA Rules not processed for user `4d7231134519451c966eca1f25e52794`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:26.715787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:26.783675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1263/2543] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:26.788148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:26.799930 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04cb1-9286-4794-a3f0-3011b68907f1 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:26.804706 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:26.805425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:26.805783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:26.806156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:26.806833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:26.808730 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:26.839230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1281/2544] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:26 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:26.840354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:26.846669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:26.854379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07ced-db01-4e65-be3d-63c9b8e085a1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:26.911730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:26.912265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:26.912548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:26.912818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:26.913992 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:26.915894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:26.939134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1264/2545] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:26 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:26.943750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:26.955810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121d09-c59d-4e8f-8b53-efe57c924f91 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:26.960729 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:26.961353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:26.961632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:26.961966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:26.962732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:26.964401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:26.976371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1282/2546] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:26.981295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:26.991741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9656d09-a3c4-49b1-9950-18a82838ca4e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:26.996315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:26.996969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:26.997255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:26.997531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] PATH_INFO: `/v3/projects/777505efae024538b68861eccda520a9/users/fde3d48f5a244a9ba99d1ba8a81d8e37/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:27.012539 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] RBAC: Authorizing `identity:create_grant(project_id=777505efae024538b68861eccda520a9, user_id=fde3d48f5a244a9ba99d1ba8a81d8e37, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:27.015234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:27.027863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1265/2547] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:26 2022] PUT /identity/v3/projects/777505efae024538b68861eccda520a9/users/fde3d48f5a244a9ba99d1ba8a81d8e37/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:27.032155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:27.042439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154ec5-8835-4a1e-a4f3-d0123bb60db5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:27.098306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:27.098925 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:27.099209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:27.099487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:27.100186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:27.101785 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c9d2f-7931-46d1-b6fd-d6ac114c4579 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:27.111416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1283/2548] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:27.115477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:27.128006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a406-5272-4a14-a94e-1717cf29977f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:27.132668 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:27.133329 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:27.133630 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:27.133919 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] PATH_INFO: `/v3/projects/777505efae024538b68861eccda520a9/users/fde3d48f5a244a9ba99d1ba8a81d8e37/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:27.135888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] RBAC: Authorizing `identity:create_grant(project_id=777505efae024538b68861eccda520a9, user_id=fde3d48f5a244a9ba99d1ba8a81d8e37, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:27.138568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:27.149300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1266/2549] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:27 2022] PUT /identity/v3/projects/777505efae024538b68861eccda520a9/users/fde3d48f5a244a9ba99d1ba8a81d8e37/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:27.849529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:27.850068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:27.850358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:27.876545 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:27.888993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] MFA Rules not processed for user `fde3d48f5a244a9ba99d1ba8a81d8e37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:27.889474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:27.993079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1284/2550] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:28.010287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:28.012346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:28.105528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c32f3b-37fa-4c08-9b54-7aba2c23401d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:28.129009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:28.129912 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:28.130095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:28.130493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:28.196906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:28.198390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:28.209261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1267/2551] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:29.579687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:29.589297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dd45b-c627-456b-9a02-2cabd8e9711f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:29.592813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:29.593422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:29.593703 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:29.594006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:29.603770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:29.605183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433aa7-8c91-4f65-8278-74356e8b7f43 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:29.614610 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1285/2552] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:29 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:38.118253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:38.119694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:38.131937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:38.196648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a33e0-6539-44c6-9a31-e0c45dd2454c service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:38.201297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:38.201963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.202248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.202520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] PATH_INFO: `/v3/users/cfe50161271849988351391d094849d0` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:38.203806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe50161271849988351391d094849d0)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:38.206507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:38.255571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe50161271849988351391d094849d0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.263417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe50161271849988351391d094849d0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.264272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe50161271849988351391d094849d0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.277350 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe50161271849988351391d094849d0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.278050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe50161271849988351391d094849d0'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.294085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1268/2553] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:38 2022] DELETE /identity/v3/users/cfe50161271849988351391d094849d0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:38.299618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.300127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.301481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:38.321431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:38.322858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:38.414814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1286/2554] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:57:38.424550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:38.426065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:38.490243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94504-9145-4623-a27a-a67ec051c27b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:38.493997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:38.494608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.494893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.495168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:38.504373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:38.505933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c7293-e96c-440c-80ec-93f11b3ecb65 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:38.515516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1269/2555] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:38.790880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:38.792311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:38.806901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500995e3-ca8f-4e96-a951-925e8ce46288 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:38.812339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:38.813702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.814022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.814302 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] PATH_INFO: `/v3/projects/7b1add18ecf94c3eb1078aaa5866a8a0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:38.815667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1add18ecf94c3eb1078aaa5866a8a0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:38.817639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:38.868320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1add18ecf94c3eb1078aaa5866a8a0', 'request_id': 'req-4499a4ac-507c-49a8-825e-f4d724cce4b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.874112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1add18ecf94c3eb1078aaa5866a8a0', 'request_id': 'req-4499a4ac-507c-49a8-825e-f4d724cce4b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.882230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1add18ecf94c3eb1078aaa5866a8a0', 'request_id': 'req-4499a4ac-507c-49a8-825e-f4d724cce4b6'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.893956 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] The token cache is being invalidate because project 7b1add18ecf94c3eb1078aaa5866a8a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:38.894245 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.894864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4499a4ac-507c-49a8-825e-f4d724cce4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:38.896373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1287/2556] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:38 2022] DELETE /identity/v3/projects/7b1add18ecf94c3eb1078aaa5866a8a0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:38.901269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.901558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.901861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:38.921038 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:38.922556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:38.949899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:38.985590 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1270/2557] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:38.993532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:38.993806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:38.994130 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.003290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:39.014960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:39.016256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:39.024388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1288/2558] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:39.028619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.039858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c463a36-85bd-4786-bc6f-2871be5a75f3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.044438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.045048 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.045333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.045609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.046349 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:39.046655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.048879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.060396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1271/2559] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 12:57:39.064674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.073970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13235432-6060-4fb3-9fce-1d6da9418ca5 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.078592 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.079724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.080002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.080272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.080963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.082886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.109681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1289/2560] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:39 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:39.114341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.124628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82cb15-bba9-416c-9aed-3c8e0fe8d85a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.182364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.182987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.183271 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.183547 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.184222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.186174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.210049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1272/2561] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:39 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:39.214754 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.227358 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a664bc-7253-4af1-8868-f9476c06cd65 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.232247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.232871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.233155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.233432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.234177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.235823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.246173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1290/2562] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:39.250387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.262824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c525a-af7c-4b59-b1ae-281af136f2bf admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.269287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.269944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.270231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.270508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.271213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.272819 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c1912-1e91-4592-85ac-d9919fb677a4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.286885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1273/2563] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 12:57:39.291361 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.301891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e833a60-55d6-49a6-aac5-1412d48f7fe5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.306941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.307622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.307916 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.308174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] PATH_INFO: `/v3/projects/c89ceca66f9345f6941b38a9de036b01/users/fe1b15615d54407dbfc749a03233039d/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.323776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89ceca66f9345f6941b38a9de036b01, user_id=fe1b15615d54407dbfc749a03233039d, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.326589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.340820 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1291/2564] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:39 2022] PUT /identity/v3/projects/c89ceca66f9345f6941b38a9de036b01/users/fe1b15615d54407dbfc749a03233039d/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:39.354354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.354696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.354980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.383248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] MFA Rules not processed for user `fe1b15615d54407dbfc749a03233039d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:39.384476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:39.457383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1274/2565] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:39.474399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.475561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:39.546558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3487-5eed-445f-96d9-bbffd61bb465 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.550877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.551672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.552016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.552362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.567071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.568712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1153fa-705f-4491-96ab-349c8e54b8dd service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.580877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1292/2566] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:39.608246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.650109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:39.696625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39cbc4-1208-4347-a8dd-72e15d2fe7ce None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.706434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.708007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.708284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.708556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] PATH_INFO: `/v3/users/fe1b15615d54407dbfc749a03233039d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.709764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1b15615d54407dbfc749a03233039d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:39.711828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:39.760311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1b15615d54407dbfc749a03233039d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:39.768243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1b15615d54407dbfc749a03233039d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:39.768927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1b15615d54407dbfc749a03233039d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:39.782732 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1b15615d54407dbfc749a03233039d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:39.783267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1b15615d54407dbfc749a03233039d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:39.797745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1275/2567] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:39 2022] DELETE /identity/v3/users/fe1b15615d54407dbfc749a03233039d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:39.802347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.802647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.802932 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:39.823246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:39.824448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:39.895647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1293/2568] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:39.917543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:39.919230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:39.989570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63317-33c1-421e-b5d8-82f67dc73ec5 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:39.993178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:39.993790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:39.994110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:39.994887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:40.004033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:40.005450 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:40.015023 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1276/2569] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:40.333620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:40.338879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:40.376523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2eab8-f21f-4dd6-80f6-edb55c571c2f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:40.398257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:40.399239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:40.399676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:40.400108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] PATH_INFO: `/v3/projects/c89ceca66f9345f6941b38a9de036b01` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:40.403627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89ceca66f9345f6941b38a9de036b01)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:40.405670 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:40.476059 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:40.490981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89ceca66f9345f6941b38a9de036b01', 'request_id': 'req-4781b138-9d0f-45a1-bfe0-f8e0164f06af'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:40.502980 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89ceca66f9345f6941b38a9de036b01', 'request_id': 'req-4781b138-9d0f-45a1-bfe0-f8e0164f06af'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:40.514540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89ceca66f9345f6941b38a9de036b01', 'request_id': 'req-4781b138-9d0f-45a1-bfe0-f8e0164f06af'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:40.527044 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] The token cache is being invalidate because project c89ceca66f9345f6941b38a9de036b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:40.527362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:40.528064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:40.541848 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1294/2570] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:40 2022] DELETE /identity/v3/projects/c89ceca66f9345f6941b38a9de036b01 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:44.701102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:44.702665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:44.715465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1b283-44c1-413f-a165-be777f792bc7 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:44.781046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:44.786199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:44.786834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:44.787687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:44.787923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:44.790537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:44.856561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:44.858045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:44.868600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1277/2571] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:45.188269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:45.189705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:45.202782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b138-9d0f-45a1-bfe0-f8e0164f06af admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:45.209664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:45.210338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:45.210625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:45.210900 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] PATH_INFO: `/v3/users/4d7231134519451c966eca1f25e52794` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:45.212071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7231134519451c966eca1f25e52794)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:45.214013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:45.263411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7231134519451c966eca1f25e52794'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.264066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7231134519451c966eca1f25e52794'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.272416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7231134519451c966eca1f25e52794'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.285951 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7231134519451c966eca1f25e52794'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.293110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7231134519451c966eca1f25e52794'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.300033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1295/2572] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:45 2022] DELETE /identity/v3/users/4d7231134519451c966eca1f25e52794 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:45.475518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:45.487502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d8827-a385-4a71-a536-2212e8f2cf5c service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:45.556664 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:45.557327 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:45.557605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:45.557918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] PATH_INFO: `/v3/projects/20fcc3e0a1f343e8835ccc9201668267` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:45.559118 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fcc3e0a1f343e8835ccc9201668267)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:45.561219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:45.607694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fcc3e0a1f343e8835ccc9201668267', 'request_id': 'req-7a889419-4429-4e58-9519-3f180db8c442'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.614414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fcc3e0a1f343e8835ccc9201668267', 'request_id': 'req-7a889419-4429-4e58-9519-3f180db8c442'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.622152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fcc3e0a1f343e8835ccc9201668267', 'request_id': 'req-7a889419-4429-4e58-9519-3f180db8c442'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.632790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] The token cache is being invalidate because project 20fcc3e0a1f343e8835ccc9201668267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:45.633333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.634007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.635553 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1278/2573] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:45 2022] DELETE /identity/v3/projects/20fcc3e0a1f343e8835ccc9201668267 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:45.639455 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:45.640920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:45.653860 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925d29d-3287-4897-ab0b-397571897c7c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:45.712872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:45.713586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:45.713882 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:45.714161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] PATH_INFO: `/v3/users/fde3d48f5a244a9ba99d1ba8a81d8e37` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:45.715388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde3d48f5a244a9ba99d1ba8a81d8e37)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:45.717357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:45.767398 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:45.787257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde3d48f5a244a9ba99d1ba8a81d8e37'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.790008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde3d48f5a244a9ba99d1ba8a81d8e37'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.800607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde3d48f5a244a9ba99d1ba8a81d8e37'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.826549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde3d48f5a244a9ba99d1ba8a81d8e37'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.835805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-db593324-6095-42be-b25f-26a7e6e8da5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde3d48f5a244a9ba99d1ba8a81d8e37'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:45.842604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1296/2574] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:45 2022] DELETE /identity/v3/users/fde3d48f5a244a9ba99d1ba8a81d8e37 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:46.041468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.043253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:46.056820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889419-4429-4e58-9519-3f180db8c442 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.087589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:46.122842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.123505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.123790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.124072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] PATH_INFO: `/v3/projects/777505efae024538b68861eccda520a9` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.125285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777505efae024538b68861eccda520a9)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.127493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.178723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777505efae024538b68861eccda520a9', 'request_id': 'req-b9b84552-081e-4408-90f0-d71d67f92d33'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:46.185124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777505efae024538b68861eccda520a9', 'request_id': 'req-b9b84552-081e-4408-90f0-d71d67f92d33'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:46.193894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777505efae024538b68861eccda520a9', 'request_id': 'req-b9b84552-081e-4408-90f0-d71d67f92d33'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:46.206264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] The token cache is being invalidate because project 777505efae024538b68861eccda520a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:46.206677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:46.207335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b9b84552-081e-4408-90f0-d71d67f92d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:46.208866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1279/2575] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:46 2022] DELETE /identity/v3/projects/777505efae024538b68861eccda520a9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:46.213661 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.214225 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.214339 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.234662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:46.236667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:46.313995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1297/2576] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:46.322128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.322417 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.322785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.345453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:46.347124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:46.367771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1280/2577] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:46.372163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.382654 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1a740-006e-42c7-9f04-66d5ef63a227 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.387991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.388612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.388895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.389170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.389923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:46.390231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.391803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.407968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1298/2578] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:57:46.409161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.418612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a20c6e-85f1-4b59-bc80-bad5a1e706cc None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.424688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.425305 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.425583 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.426310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.426536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.428487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.465669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1281/2579] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:46 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:46.466936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.481518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90963930-f04a-4b54-8651-2aa569580383 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.569158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.569832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.570131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.570405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.571068 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.572973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.597976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1299/2580] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:46 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:46.602869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.615432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659766-cc59-4921-bed9-7870706db291 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.620525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.621360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.621748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.622168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.623222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.625733 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.641869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1282/2581] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:46.643779 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.659371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f8ecf-8d36-42df-8e9d-bb8baa387d5f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.664945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.665582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.665952 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.666603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.667001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.669676 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d479f2-154d-44bd-871f-f660fd21afa0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.682535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1300/2582] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:46.688117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.701014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62240f83-c880-4b52-8d44-827b20d3f248 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:46.709168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:46.710183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.710829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.711372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] PATH_INFO: `/v3/projects/c3a439dae781431fa3253d60bcefceb0/users/1f2373e74c164e47bcdbf30a6b8d8fff/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.731601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a439dae781431fa3253d60bcefceb0, user_id=1f2373e74c164e47bcdbf30a6b8d8fff, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:46.734498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:46.749395 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1283/2583] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:46 2022] PUT /identity/v3/projects/c3a439dae781431fa3253d60bcefceb0/users/1f2373e74c164e47bcdbf30a6b8d8fff/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:46.818963 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:46.819253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:46.819535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:46.868072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] MFA Rules not processed for user `1f2373e74c164e47bcdbf30a6b8d8fff`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:46.869330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:46.882911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:46.949145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1301/2584] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:46.960691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:46.962555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:47.031051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab44db-3a9a-4c3c-b558-1f0b2689cc2c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:47.037302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:47.037952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.038231 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.038503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:47.048205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:47.049636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc693d28-d105-40de-8e9e-9358043e69c2 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:47.054408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:47.060352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1284/2585] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:47.079595 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:47.150765 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3dbb-0922-40e4-82e2-ac0224743f80 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:47.155767 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:47.156488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.157352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.157698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] PATH_INFO: `/v3/users/1f2373e74c164e47bcdbf30a6b8d8fff` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:47.158989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2373e74c164e47bcdbf30a6b8d8fff)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:47.161004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:47.226191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2373e74c164e47bcdbf30a6b8d8fff'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.227602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2373e74c164e47bcdbf30a6b8d8fff'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.237011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2373e74c164e47bcdbf30a6b8d8fff'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.255398 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2373e74c164e47bcdbf30a6b8d8fff'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.266326 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2373e74c164e47bcdbf30a6b8d8fff'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.276370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1302/2586] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:47 2022] DELETE /identity/v3/users/1f2373e74c164e47bcdbf30a6b8d8fff => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:47.278055 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.278370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.278798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:47.302009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:47.303399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:47.385081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1285/2587] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 12:57:47.395268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:47.396751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:47.442586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:47.471140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a7e75-a95f-4a78-b19b-7c8b818fb5a7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:47.475231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:47.475886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.476168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.476445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:47.487626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:47.489067 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be979e75-77c3-4992-a1dd-01c5287cdf68 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:47.499961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1303/2588] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:47.829913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:47.831423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:47.845479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba2c33-a82a-4d2f-ab8e-132c365e81a0 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:47.851917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:47.852792 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.853056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.853414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] PATH_INFO: `/v3/projects/c3a439dae781431fa3253d60bcefceb0` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:47.854714 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a439dae781431fa3253d60bcefceb0)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:47.856979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:47.919581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a439dae781431fa3253d60bcefceb0', 'request_id': 'req-fc7ecfed-f36c-4901-906b-7c730506ca3d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.931165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a439dae781431fa3253d60bcefceb0', 'request_id': 'req-fc7ecfed-f36c-4901-906b-7c730506ca3d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.947872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a439dae781431fa3253d60bcefceb0', 'request_id': 'req-fc7ecfed-f36c-4901-906b-7c730506ca3d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.967823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] The token cache is being invalidate because project c3a439dae781431fa3253d60bcefceb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:47.968216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.968969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:47.971500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1286/2589] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:47 2022] DELETE /identity/v3/projects/c3a439dae781431fa3253d60bcefceb0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:47.977490 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de789798-24cd-484e-97dc-d9b17400ede8 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:47.977783 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de789798-24cd-484e-97dc-d9b17400ede8 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:47.978090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de789798-24cd-484e-97dc-d9b17400ede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.021569 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-de789798-24cd-484e-97dc-d9b17400ede8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:48.023224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-de789798-24cd-484e-97dc-d9b17400ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.043140 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.045763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.105319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1304/2590] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:48.116295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.116594 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.116873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.123796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ecfed-f36c-4901-906b-7c730506ca3d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.129320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.129947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.130303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.130648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] PATH_INFO: `/v3/users/e7ee2c3480f8483bb76ecc686215720b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.131917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ee2c3480f8483bb76ecc686215720b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.134001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.137350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:48.138582 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.146577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1305/2591] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:48.150596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.157436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:48.159802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c5055-b4b7-4990-a6b8-d38101f6fc4e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.165430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.166065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.166370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.166647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.167362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:48.167656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.169307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.181794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1306/2592] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:57:48.186043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.186226 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ee2c3480f8483bb76ecc686215720b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.193470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ee2c3480f8483bb76ecc686215720b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.194175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ee2c3480f8483bb76ecc686215720b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.195774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8ba5-f520-4a9d-beec-48e11ea23ab9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.200370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.200960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.201244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.201518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.202201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.204623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.206929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ee2c3480f8483bb76ecc686215720b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.207570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ee2c3480f8483bb76ecc686215720b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.226409 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1287/2593] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 12:57:48 2022] DELETE /identity/v3/users/e7ee2c3480f8483bb76ecc686215720b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:48.236738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1307/2594] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 12:57:48.238295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.239686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.305866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bf418-5371-499a-a536-77021d34537b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.360810 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.361425 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.361704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.362000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.362653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.364557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.388339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1288/2595] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:48.393540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.402865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.404685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.405506 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6bd99-2c86-4900-ad00-dfd2734b36c7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.412243 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.413143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.413550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.413964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.414955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.417159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.419853 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627e0d-e454-4986-8389-8e4c453e1ad6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.425770 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.426832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.426832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.427485 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] PATH_INFO: `/v3/projects/89441418bb3d4ff6ad1a0b466e69cfb6` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.429472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89441418bb3d4ff6ad1a0b466e69cfb6)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.431710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.432986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1308/2596] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:48.437122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.449023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c31a00-c2b5-4ad5-9abb-01cecbd7b8e7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.455611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.456712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.457442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.457931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.458816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.460718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.477636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1309/2597] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:48.478591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.480622 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:48.490832 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85eb6c-e476-412a-8815-5358bc1e8af2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.506150 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89441418bb3d4ff6ad1a0b466e69cfb6', 'request_id': 'req-095fff30-70a2-486f-b9db-afb6aee1bff4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.516613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89441418bb3d4ff6ad1a0b466e69cfb6', 'request_id': 'req-095fff30-70a2-486f-b9db-afb6aee1bff4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.526211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89441418bb3d4ff6ad1a0b466e69cfb6', 'request_id': 'req-095fff30-70a2-486f-b9db-afb6aee1bff4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.540619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] The token cache is being invalidate because project 89441418bb3d4ff6ad1a0b466e69cfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 12:57:48.540990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.541835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 12:57:48.546066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1289/2598] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 12:57:48 2022] DELETE /identity/v3/projects/89441418bb3d4ff6ad1a0b466e69cfb6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:48.546066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.546307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.558702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.559384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.559680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.560032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] PATH_INFO: `/v3/projects/700a0e9f82e74d2fa8ab7ae512864b7f/users/4896dca922614700822ab9ba440c504a/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.579561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] RBAC: Authorizing `identity:create_grant(project_id=700a0e9f82e74d2fa8ab7ae512864b7f, user_id=4896dca922614700822ab9ba440c504a, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.582312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd88f76-d061-4b04-abd9-50d2a6b4680d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.585562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:48.601172 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1310/2599] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:48 2022] PUT /identity/v3/projects/700a0e9f82e74d2fa8ab7ae512864b7f/users/4896dca922614700822ab9ba440c504a/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:48.602524 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6dec-ef1c-4075-aeaa-8bb958952099 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.602812 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6dec-ef1c-4075-aeaa-8bb958952099 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.603085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6dec-ef1c-4075-aeaa-8bb958952099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.620640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fff30-70a2-486f-b9db-afb6aee1bff4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.630660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-8a3b6dec-ef1c-4075-aeaa-8bb958952099 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:48.631865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-8a3b6dec-ef1c-4075-aeaa-8bb958952099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.670011 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.670629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.670910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.671179 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.673009 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.691708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1311/2600] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:48.695713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.696013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.696292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.729433 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] MFA Rules not processed for user `4896dca922614700822ab9ba440c504a`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:48.730674 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.734663 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.736785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd168a-c2e2-42db-adbe-e4e4734a280d service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.746453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1290/2601] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:48.747755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.748117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.748470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.770279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:48.771501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.780157 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1291/2602] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:48.784354 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.793933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6560f2-a963-46b0-83d5-152eff1ec1bb None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.799785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.800172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.800457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.800760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.801473 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 12:57:48.801756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.803374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.808607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1312/2603] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:48.822540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1292/2604] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 12:57:48.822736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.822736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:48.824946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.835442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285271e-b49f-48a4-853a-de89d4ada8cf None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.836343 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2f45d-d2bc-4e99-8dd3-e090c6ea2fb1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.840516 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.841221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.841422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.841693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.842439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.843386 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.843671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.844197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.844901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.847268 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.852013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.854458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:48.858551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:48.871266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1313/2605] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 12:57:48.883434 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1293/2606] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/projects => generated 357 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:48.884246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:48.896550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d98ba-75b1-442a-88dd-c1285ee2eed9 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:48.974381 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:48.975276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:48.975659 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:48.976029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:48.976798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:48.978886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.005101 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1314/2607] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:48 2022] POST /identity/v3/users => generated 279 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:49.006436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.018447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2cc5f-cbaa-4fee-acab-1f9e25cfb771 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.023709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.024405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.024754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.025082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.025778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.027619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.038957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1294/2608] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:49.047162 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.054291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:49.062204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14cf36-a2fd-4510-90c0-cea1c62d8e10 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.067553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.068315 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.068906 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.069187 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.070427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.073577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c11589-f6e3-4fbd-a563-7b2f62dcbd44 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.085292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1315/2609] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:49.089489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.100320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8206d8-4aa4-458a-960f-cbf9d6751194 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.105632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.106363 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.106656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.106987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] PATH_INFO: `/v3/projects/29ee5aa813774cee8b0c2f3cfccb085e/users/ab328225c00141a0a55ef4b24653851b/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.125211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ee5aa813774cee8b0c2f3cfccb085e, user_id=ab328225c00141a0a55ef4b24653851b, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.128387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.142982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1295/2610] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:49 2022] PUT /identity/v3/projects/29ee5aa813774cee8b0c2f3cfccb085e/users/ab328225c00141a0a55ef4b24653851b/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:49.153692 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.154017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.154517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.189384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] MFA Rules not processed for user `ab328225c00141a0a55ef4b24653851b`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:49.190823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:49.267733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1316/2611] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:49 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:49.272355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.331895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27514-2daa-404b-a408-b29d50187ed9 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.336469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.337071 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.337352 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.337626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.338314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.340200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.365934 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1296/2612] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 12:57:49 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:49.370510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.381587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba9282-bbd1-4221-bcb6-ee5430793a7f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.436943 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.437561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.438387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.438666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.439320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.441323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.462469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1317/2613] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 12:57:49 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 12:57:49.466490 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.477170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1a32c-1cc8-4022-9182-dd4b65a6ee6b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.481729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.482366 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.482649 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.482923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.483613 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.485199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.495524 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1297/2614] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:49.499451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.509801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56c4bd-1d62-4266-8eaf-3220a14a461d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.514387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.515041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.515325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.515601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] PATH_INFO: `/v3/projects/3733055d57b64620b51b247ae22aa958/users/804d229d584e43fb900f5b0a263707d3/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.530829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3733055d57b64620b51b247ae22aa958, user_id=804d229d584e43fb900f5b0a263707d3, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.533565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.546465 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1318/2615] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:49 2022] PUT /identity/v3/projects/3733055d57b64620b51b247ae22aa958/users/804d229d584e43fb900f5b0a263707d3/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 12:57:49.550598 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.561684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb99ff3-9f3f-4bf6-9aec-899f5bd4aa17 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.617642 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.618210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.618488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.618759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.619439 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.621016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8247d6b-15cb-4fa8-9905-0ac66f5fd1c3 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.631065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1298/2616] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 12:57:49.635244 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.645432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c76a-81b0-4ba0-97f3-fa274dad82d9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.649982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.650636 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.650915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.651189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] PATH_INFO: `/v3/projects/3733055d57b64620b51b247ae22aa958/users/804d229d584e43fb900f5b0a263707d3/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.653185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3733055d57b64620b51b247ae22aa958, user_id=804d229d584e43fb900f5b0a263707d3, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.655844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.665699 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1319/2617] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 12:57:49 2022] PUT /identity/v3/projects/3733055d57b64620b51b247ae22aa958/users/804d229d584e43fb900f5b0a263707d3/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 12:57:49.677631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.677949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.678222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.706051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] MFA Rules not processed for user `804d229d584e43fb900f5b0a263707d3`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 12:57:49.707251 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:57:49.718868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:49.779593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1299/2618] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 12:57:49 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 12:57:49.791057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.826161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:57:49.847750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b43f0-69e1-403c-85d1-47c9195d079f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.851277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.851880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.852186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.852459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:49.861713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:49.863156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:49.872366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1320/2619] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:49.946635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:57:49.956239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a5774-f8b8-4c4b-802b-83bb2b67a364 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:57:49.960014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:57:49.960624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:57:49.960900 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:57:49.961170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:57:50.019878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:57:50.021312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:57:50.030431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1300/2620] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:57:49 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:57:59.868206 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=11 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:58:00.026110 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=51 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 12:58:39.302910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 12:58:39.304368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:58:39.456889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1c1ac-eb22-45f8-b2b7-8f9032162af6 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 12:58:39.461881 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 12:58:39.462499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 12:58:39.462782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 12:58:39.463057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 12:58:39.464811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 12:58:39.476771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 12:58:39.478353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 12:58:39.488628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1321/2621] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 12:58:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 12:58:49.484543 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=63 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:01:31.745336 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:01:31.746888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:01:31.833534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7fa20-e527-449a-a34e-244d06c14538 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:01:31.837133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:01:31.837741 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:01:31.838222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:01:31.839086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:01:31.840827 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:01:31.850404 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:01:31.851816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:01:31.859794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1301/2622] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:01:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:01:41.856181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=40 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:01:48.760352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:01:48.780318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416eeb5-444a-4677-864b-9a9a8d038c25 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:01:48.783969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:01:48.784578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:01:48.784859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:01:48.785135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:01:48.786905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:01:48.868236 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:01:48.870300 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:01:48.888340 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1322/2623] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:01:58.885681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=45 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:02:49.936556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:02:49.946315 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40a6cb-6585-401d-9f65-4057e9e2f7e6 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:02:49.950087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:02:49.950700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:02:49.951010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:02:49.951285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:02:50.016378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:02:50.017836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:02:50.036269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1302/2624] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:02:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:02:50.062422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:02:50.071141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808638-1ec1-4d63-931d-f5cf36a3169f service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:02:50.074773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:02:50.075384 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:02:50.075667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:02:50.075946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:02:50.133853 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:02:50.135392 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:02:50.144950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1323/2625] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:02:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:02:50.910676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:02:50.919373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95cd93-a273-4f96-866a-31cc80f817b7 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:02:50.922932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:02:50.923526 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:02:50.923805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:02:50.924075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:02:50.932916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:02:50.934494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b074e7-30c0-4d78-9f8c-4398af3307a0 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:02:50.944074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1303/2626] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:02:50 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:03:00.140637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=28 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:03:00.939962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=42 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:03:49.110261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:03:49.115167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:03:49.220546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded6e04-cc26-485c-959e-9008aa3d740c service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:03:49.225242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:03:49.225909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:03:49.226221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:03:49.226501 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] PATH_INFO: `/v3/users/4896dca922614700822ab9ba440c504a` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:03:49.228550 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4896dca922614700822ab9ba440c504a)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:03:49.230673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:03:49.276210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4896dca922614700822ab9ba440c504a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.276862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4896dca922614700822ab9ba440c504a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.283913 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4896dca922614700822ab9ba440c504a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.296484 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4896dca922614700822ab9ba440c504a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.303366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4896dca922614700822ab9ba440c504a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.309462 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1324/2627] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:03:49 2022] DELETE /identity/v3/users/4896dca922614700822ab9ba440c504a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:03:49.313989 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:03:49.314478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:03:49.314587 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:03:49.344863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:03:49.346102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:03:49.403600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1304/2628] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:03:49.413413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:03:49.414886 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:03:49.471057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5796456-0390-4b68-b38a-b95ddbfbeb05 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:03:49.474685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:03:49.475296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:03:49.475579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:03:49.475856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:03:49.481938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:03:49.487042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:03:49.489311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:03:49.499987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1325/2629] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:03:49.767987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:03:49.769522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:03:49.781247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c64dd-9d65-4586-bf49-f379cb21bf11 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:03:49.786895 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:03:49.787892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:03:49.788183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:03:49.788460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] PATH_INFO: `/v3/projects/700a0e9f82e74d2fa8ab7ae512864b7f` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:03:49.789631 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] RBAC: Authorizing `identity:delete_project(project_id=700a0e9f82e74d2fa8ab7ae512864b7f)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:03:49.792961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:03:49.842029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700a0e9f82e74d2fa8ab7ae512864b7f', 'request_id': 'req-adf042d3-ebf0-4024-8b17-7dcca728685a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.848142 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700a0e9f82e74d2fa8ab7ae512864b7f', 'request_id': 'req-adf042d3-ebf0-4024-8b17-7dcca728685a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.856248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700a0e9f82e74d2fa8ab7ae512864b7f', 'request_id': 'req-adf042d3-ebf0-4024-8b17-7dcca728685a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.866467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] The token cache is being invalidate because project 700a0e9f82e74d2fa8ab7ae512864b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:03:49.866754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.867396 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:03:49.869515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1305/2630] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:03:49 2022] DELETE /identity/v3/projects/700a0e9f82e74d2fa8ab7ae512864b7f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:03:59.495572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:03:59.863442 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:04:28.069510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:04:28.071632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:04:28.144634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb8a81-68c3-44a9-82a3-5af6b6d4592f service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:04:28.213559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:04:28.214214 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:04:28.214494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:04:28.214768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:04:28.216481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:04:28.228031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:04:28.229870 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:04:28.239479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1326/2631] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:04:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:04:38.235324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=58 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:04:38.881080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:04:38.883496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:04:38.953500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf042d3-ebf0-4024-8b17-7dcca728685a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:04:38.956988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:04:38.957601 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:04:38.957922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:04:38.958201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:04:38.959955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:04:39.050176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:04:39.051612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:04:39.071926 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1306/2632] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:04:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:04:49.068818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=71 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:06:34.736606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:06:34.738083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:06:34.796199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b850df-1670-4988-ac2c-ab0a7a994131 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:06:34.799771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:06:34.800391 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:06:34.800675 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:06:34.800955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:06:34.803287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:06:34.812835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:06:34.814311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:06:34.822437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1327/2633] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:06:44.819048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=26 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:06:49.918502 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:06:49.920022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:06:49.934374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f4f3c-7b52-45c8-a7f7-c385a209de50 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:06:49.939760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:06:49.940365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:06:49.940644 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:06:49.940917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:06:49.942607 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:06:49.999935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:06:50.001494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:06:50.011035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1307/2634] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:06:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:07:00.006844 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=28 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:07:50.937283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:07:50.946500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550966c-9e3e-4b62-8ba6-9294fcc7ffa6 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:07:50.950275 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:07:50.950890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:07:50.951173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:07:50.951449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:07:50.953184 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:07:51.023357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:07:51.024715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:07:51.033600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1328/2635] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:07:51.951345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:07:51.961173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2be0-5df7-4ef9-894e-35d24e1592fe service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:07:51.964697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:07:51.965300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:07:51.965567 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:07:51.965861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:07:51.967599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:07:52.044952 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:07:52.046448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:07:52.064256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1308/2636] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:08:01.029602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=33 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:08:02.061410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=41 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:09:33.028579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:09:33.030005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:33.109492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab4d9d-14fe-4fe9-bd17-4d6f448b22bc service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:09:33.115357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:09:33.115992 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:09:33.116263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:09:33.116679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:09:33.126668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:09:33.127873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:09:33.137892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1329/2637] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:09:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:09:43.133562 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=32 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:09:50.101864 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:09:50.136756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924781d-9d1b-4302-83cd-95a656cb777c service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:09:50.142953 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:09:50.143575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:09:50.143856 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:09:50.144204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:09:50.145898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:50.160014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:09:50.161748 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:09:50.173253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1309/2638] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:09:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:09:50.292920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:09:50.307521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4e2c-2afa-4dee-ab9a-1ef3393afec7 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:09:50.311941 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:09:50.312684 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:09:50.313037 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:09:50.313496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:09:50.325701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:09:50.327294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:09:50.341268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1330/2639] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:09:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:09:51.446047 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:09:51.447978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:51.464411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2e79-335b-4c43-a350-e7e6db81073d service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:09:51.471339 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:09:51.471955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:09:51.472236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:09:51.472510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:09:51.474269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:51.487872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:09:51.489454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:09:51.501180 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1310/2640] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:09:51.606363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:09:51.607759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:51.620079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea377c-5a90-489b-9b2b-8304cbfc2912 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:09:51.623701 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:09:51.624304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:09:51.624572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:09:51.624849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:09:51.626660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:09:51.639605 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:09:51.640946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:09:51.652208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1331/2641] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:10:01.495593 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=18 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:10:01.647355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=12 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:11:40.748246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:11:40.777317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541135b-ea61-4df3-a4f7-b78a8c4d77c2 service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:11:40.781511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:11:40.782269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:11:40.782552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:11:40.782823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:11:40.790187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:11:40.791602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:11:40.800230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1311/2642] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:11:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:11:50.796112 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=15 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:11:51.512183 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:11:51.513529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:11:51.544958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94d442-893d-4a9b-9e6f-b318e810e290 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:11:51.548638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:11:51.549242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:11:51.549520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:11:51.549792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:11:51.551507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:11:51.634611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:11:51.636158 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:11:51.655972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1332/2643] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:11:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:11:51.683170 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:11:51.692345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c7f91-a494-4432-892e-27749c78f92e service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:11:51.695908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:11:51.696503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:11:51.696778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:11:51.697052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:11:51.705906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:11:51.707312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:11:51.734899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1312/2644] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:11:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:12:01.652630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=52 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:12:01.732253 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=13 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:12:51.315625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:12:51.328224 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c9dbc-993a-42b5-85a1-822d994c5d63 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:12:51.334297 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:12:51.334914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:12:51.335195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:12:51.335468 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:12:51.343387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:12:51.344962 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:12:51.372814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1333/2645] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:12:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:13:01.370146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=13 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:14:38.019918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:14:38.021384 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:14:38.094265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdfd3c-e5f5-4d7d-b131-90fe61aeb75f service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:14:38.099435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:14:38.100050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:14:38.100355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:14:38.100655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:14:38.102620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:14:38.115264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:14:38.116865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:14:38.127002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1313/2646] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:14:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:14:48.122681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=32 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:14:50.519395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:14:50.528581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c68b-4893-45bb-ba81-a1ab63dec7ef service nova] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:14:50.532153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:14:50.532761 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:14:50.533040 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:14:50.533311 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:14:50.535309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:14:50.622734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:14:50.624163 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:14:50.645623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1334/2647] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:14:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:14:51.752796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:14:51.762087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439717e-3d21-437a-9125-6864c1c24c4e service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:14:51.765612 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:14:51.766248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:14:51.766529 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:14:51.766802 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:14:51.777496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:14:51.778957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140f10b-af56-438a-8bb2-563ae63d04e3 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:14:51.790428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1314/2648] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:14:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:15:00.642411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=46 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:01.785608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=6 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:52.179287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:52.180809 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:52.281062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d042f3-b969-450c-8dab-7a877c79fd0b service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:52.286466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:52.287122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:52.287434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:52.287732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] PATH_INFO: `/v3/users/ab328225c00141a0a55ef4b24653851b` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:52.288939 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab328225c00141a0a55ef4b24653851b)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:52.291092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:52.336572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab328225c00141a0a55ef4b24653851b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.337211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab328225c00141a0a55ef4b24653851b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.344328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab328225c00141a0a55ef4b24653851b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.357096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab328225c00141a0a55ef4b24653851b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.364485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab328225c00141a0a55ef4b24653851b'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.370430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1335/2649] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:15:52 2022] DELETE /identity/v3/users/ab328225c00141a0a55ef4b24653851b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:52.374915 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:52.375432 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:52.375550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:52.402815 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:15:52.404001 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:52.461611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1315/2650] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:15:52.472017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:52.473417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:52.528581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:52.545105 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e77d5-904e-4bb6-ab1a-25bdeaf00e7b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:52.548800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:52.549417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:52.549695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:52.550013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:52.558873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:52.560287 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:52.569512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1336/2651] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:15:52.836385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:52.838203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:52.852918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c4e45-03c6-4dc0-a0bb-b67465816a05 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:52.857730 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:52.858552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:52.858896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:52.859258 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] PATH_INFO: `/v3/projects/29ee5aa813774cee8b0c2f3cfccb085e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:52.861413 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ee5aa813774cee8b0c2f3cfccb085e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:52.863465 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:52.910122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ee5aa813774cee8b0c2f3cfccb085e', 'request_id': 'req-15f987e0-46c5-40b9-9602-dffdc7689bef'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.916093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ee5aa813774cee8b0c2f3cfccb085e', 'request_id': 'req-15f987e0-46c5-40b9-9602-dffdc7689bef'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.924158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ee5aa813774cee8b0c2f3cfccb085e', 'request_id': 'req-15f987e0-46c5-40b9-9602-dffdc7689bef'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.934462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] The token cache is being invalidate because project 29ee5aa813774cee8b0c2f3cfccb085e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:15:52.934753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.935380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:52.936870 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1316/2652] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:15:52 2022] DELETE /identity/v3/projects/29ee5aa813774cee8b0c2f3cfccb085e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:52.941419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:52.951181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af04f0-ea14-4a7b-a63b-5cda764d7862 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.007885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.008533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.008835 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.009170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] PATH_INFO: `/v3/users/804d229d584e43fb900f5b0a263707d3` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.010387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=804d229d584e43fb900f5b0a263707d3)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.012322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.065617 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804d229d584e43fb900f5b0a263707d3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.066376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804d229d584e43fb900f5b0a263707d3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.077217 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804d229d584e43fb900f5b0a263707d3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.092116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804d229d584e43fb900f5b0a263707d3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.099319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2bc0b01b-15ba-4753-aa03-158b791e1924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804d229d584e43fb900f5b0a263707d3'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.105445 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1337/2653] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:15:52 2022] DELETE /identity/v3/users/804d229d584e43fb900f5b0a263707d3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:53.377835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.379276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:53.392210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f987e0-46c5-40b9-9602-dffdc7689bef admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.450115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.450776 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.451107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.451328 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] PATH_INFO: `/v3/projects/3733055d57b64620b51b247ae22aa958` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.452486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3733055d57b64620b51b247ae22aa958)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.454422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.475211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:53.501591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3733055d57b64620b51b247ae22aa958', 'request_id': 'req-1719ca3f-0892-4ea9-9128-463c59fd8369'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.508314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3733055d57b64620b51b247ae22aa958', 'request_id': 'req-1719ca3f-0892-4ea9-9128-463c59fd8369'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.515016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3733055d57b64620b51b247ae22aa958', 'request_id': 'req-1719ca3f-0892-4ea9-9128-463c59fd8369'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.525599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] The token cache is being invalidate because project 3733055d57b64620b51b247ae22aa958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:15:53.525920 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.526537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-1719ca3f-0892-4ea9-9128-463c59fd8369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:15:53.528033 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1317/2654] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:15:53 2022] DELETE /identity/v3/projects/3733055d57b64620b51b247ae22aa958 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:53.534948 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.535449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.535561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.583935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:15:53.585165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:53.620773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:53.648404 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1338/2655] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:15:53.658005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.658949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.659260 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.679196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:15:53.680392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:53.688236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1318/2656] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:15:53.692463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.701972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241fc4e-d0bf-4afe-b127-9e3aa8c6d316 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.706430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.707034 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.707313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.707586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.708276 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:15:53.708580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.710205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.722829 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1339/2657] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:15:53.728184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.737987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b736-28d6-4dbc-b9cf-a811fc246403 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.743197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.743798 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.744081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.744355 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.745012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.746932 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.783704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1319/2658] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:15:53 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:15:53.787816 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.797715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7b4b1-abf1-46f5-97e4-709ef6cf762a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.854320 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.854931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.855220 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.855497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.856159 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.858090 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.879612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1340/2659] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:15:53 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:15:53.885205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.896030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eac64-d5a3-4ca1-9be9-df8e2e7bcde6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.904820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.906299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.906615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.906893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.907599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.909208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.920077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1320/2660] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:15:53.924743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.938062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619d91-b393-482b-90de-7f0fbbce26c6 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.944416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.945029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.945312 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.945586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.946325 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.947904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4042-1e24-4a05-8257-c2dc72adb99b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:53.958705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1341/2661] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:15:53.962817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:53.972885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1103841-5f02-45eb-8d8c-561acf13bc01 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:53.977491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:53.978173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:53.978455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:53.978726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] PATH_INFO: `/v3/projects/dae006e435104252b408c344ba1bab85/users/1f3343d49bbd4018899079222247a6e8/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:53.993788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae006e435104252b408c344ba1bab85, user_id=1f3343d49bbd4018899079222247a6e8, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:53.996436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.009913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1321/2662] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:15:53 2022] PUT /identity/v3/projects/dae006e435104252b408c344ba1bab85/users/1f3343d49bbd4018899079222247a6e8/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:54.076407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.076694 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.076974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.103888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] MFA Rules not processed for user `1f3343d49bbd4018899079222247a6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:15:54.105118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:54.168607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1342/2663] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:15:54.173105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.239823 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac23b8-ae75-4811-8a4f-c48a21bcfe12 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.248086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.248745 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.248999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.249274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.250004 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.252446 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.280066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1322/2664] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:15:54 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:15:54.284218 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.295086 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40d9d3-78c6-47b8-82d5-99cc9972678f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.324872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:54.352583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.353202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.353487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.353764 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.354466 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.356364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.378438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1343/2665] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:15:54 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:15:54.382411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.393299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9160f-5efc-4c42-81f2-6f3c9cbb7606 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.397862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.398472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.398753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.399026 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.399718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.401302 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.411687 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1323/2666] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:15:54.415762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.425707 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c77b82-0200-404c-a477-b86629230603 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.430628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.431319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.431599 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.431893 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] PATH_INFO: `/v3/projects/fb7a1197a91f4926be74c4c06701b738/users/093c0b58d5c74349af5a23155146517e/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.447192 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7a1197a91f4926be74c4c06701b738, user_id=093c0b58d5c74349af5a23155146517e, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.449961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.462722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1344/2667] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:15:54 2022] PUT /identity/v3/projects/fb7a1197a91f4926be74c4c06701b738/users/093c0b58d5c74349af5a23155146517e/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:15:54.466678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.476707 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f9f6f-2d4d-4c74-a7e3-19e2a2543d19 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.481256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:15:54.534511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.535049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.535331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.535628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.536310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.537916 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bc843-580f-44b1-bd8b-3ee6110f325b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.547592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1324/2668] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:15:54.552007 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:15:54.562402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c044f2-2ef5-4925-922d-63c68549864f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:15:54.567491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:15:54.568166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.568449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.568727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] PATH_INFO: `/v3/projects/fb7a1197a91f4926be74c4c06701b738/users/093c0b58d5c74349af5a23155146517e/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.570837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7a1197a91f4926be74c4c06701b738, user_id=093c0b58d5c74349af5a23155146517e, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:15:54.573491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:15:54.585292 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1345/2669] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:15:54 2022] PUT /identity/v3/projects/fb7a1197a91f4926be74c4c06701b738/users/093c0b58d5c74349af5a23155146517e/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:15:54.595201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:15:54.595496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:15:54.595778 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:15:54.625043 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] MFA Rules not processed for user `093c0b58d5c74349af5a23155146517e`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:15:54.626320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:15:54.695133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1325/2670] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:16:04.581414 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=29,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:16:04.692060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=59 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:16:40.596861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:16:40.598619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:16:40.671429 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea747-b88d-4649-8e2f-4325d92d9be0 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:16:40.674953 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:16:40.675564 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:16:40.675842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:16:40.676113 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:16:40.684731 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:16:40.686186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:16:40.696144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1346/2671] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:16:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:16:41.240133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:16:41.241592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:16:41.299737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1f662-1a56-4b99-82ce-3baf9ebc5721 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:16:41.303274 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:16:41.303887 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:16:41.304765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:16:41.305078 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:16:41.306831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:16:41.316246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:16:41.317685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:16:41.325736 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1326/2672] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:16:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:16:50.691250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=31 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:16:51.322212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=26 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:17:52.290961 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:17:52.292496 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:17:52.303979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0abbe-e06e-4a22-be9c-bec48e124dac service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:17:52.307625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:17:52.308249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:17:52.308531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:17:52.308824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:17:52.310514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:17:52.382180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:17:52.384093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:17:52.392927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1347/2673] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:17:52.458640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:17:52.468711 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2bf0-4497-428f-ac24-8db09030ce5e service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:17:52.472356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:17:52.472977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:17:52.473284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:17:52.473560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:17:52.475382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:17:52.549212 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:17:52.550772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:17:52.560471 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1327/2674] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:18:02.389212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=33 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:18:02.555765 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=33 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:18:40.838899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:18:40.848735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c533e9b-c403-44ea-ad12-cd8dac420186 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:18:40.853005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:18:40.853704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:18:40.854092 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:18:40.854431 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:18:40.909704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:18:40.911801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:18:40.920548 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1348/2675] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:18:50.916634 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=28 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:19:40.015988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:19:40.017759 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:19:40.134845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e123ad4-ab6e-45d2-a26c-bd34d298d020 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:19:40.139882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:19:40.140497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:19:40.140779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:19:40.141051 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:19:40.142881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:19:40.154928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:19:40.156494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:19:40.166515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1328/2676] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:19:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:19:50.162221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=52 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:21:47.734228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:47.753386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88a2d9-b6ae-43f7-a1e7-bfa4463c1f75 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:47.757012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:47.757620 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:47.757937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:47.758710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:47.765993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:47.767411 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:47.775897 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1349/2677] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:21:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:21:52.798263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:52.818878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65ce33-60de-4020-ac9a-8b468ad15142 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:52.823381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:52.824072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:52.824357 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:52.824635 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:52.826501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:21:52.902610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:52.903962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:52.914031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1329/2678] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:21:55.795734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:55.838061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87516ca-bff0-427b-8dff-7bee6df20c73 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:55.849337 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:55.850200 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:55.851026 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:55.851525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:55.854108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:21:55.874237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:55.876135 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:55.895073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1350/2679] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:21:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:21:59.045791 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:59.047382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:21:59.061503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701fff2-7d9e-4596-a67b-1ae8234cbcd0 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:59.066641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:59.067298 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:59.067597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:59.067866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] PATH_INFO: `/v3/users/07220936c8784339b62f22482264e234` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:59.069187 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07220936c8784339b62f22482264e234)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:59.071172 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:59.129563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07220936c8784339b62f22482264e234'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.136828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07220936c8784339b62f22482264e234'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.137514 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07220936c8784339b62f22482264e234'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.149614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07220936c8784339b62f22482264e234'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.150285 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07220936c8784339b62f22482264e234'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.163079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1330/2680] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:21:59 2022] DELETE /identity/v3/users/07220936c8784339b62f22482264e234 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:21:59.363585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:59.376709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1416-0b96-47be-a5cc-a1b349da9863 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:59.435041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:59.435689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:59.435977 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:59.436257 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] PATH_INFO: `/v3/projects/15f411d81f95464389c9ea3c12cee88d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:59.437419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f411d81f95464389c9ea3c12cee88d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:59.439359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:59.485167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f411d81f95464389c9ea3c12cee88d', 'request_id': 'req-87ccfef6-ab83-4891-bcc5-c2b850e32e53'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.490822 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f411d81f95464389c9ea3c12cee88d', 'request_id': 'req-87ccfef6-ab83-4891-bcc5-c2b850e32e53'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.498213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f411d81f95464389c9ea3c12cee88d', 'request_id': 'req-87ccfef6-ab83-4891-bcc5-c2b850e32e53'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.509427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] The token cache is being invalidate because project 15f411d81f95464389c9ea3c12cee88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:21:59.509715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.510386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.517216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1351/2681] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:21:59 2022] DELETE /identity/v3/projects/15f411d81f95464389c9ea3c12cee88d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:21:59.517360 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:59.517556 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:21:59.531392 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dbe94-6b55-4ecb-846e-a1b1dc1302b6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:59.578345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:21:59.597058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:59.597701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:59.598030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:59.598318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] PATH_INFO: `/v3/users/30243c3bccab4a739cfbe1ff208c1af3` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:59.611704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30243c3bccab4a739cfbe1ff208c1af3)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:59.613558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:21:59.659572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30243c3bccab4a739cfbe1ff208c1af3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.666301 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30243c3bccab4a739cfbe1ff208c1af3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.666936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30243c3bccab4a739cfbe1ff208c1af3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.680428 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30243c3bccab4a739cfbe1ff208c1af3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.681126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30243c3bccab4a739cfbe1ff208c1af3'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:21:59.696254 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1331/2682] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:21:59 2022] DELETE /identity/v3/users/30243c3bccab4a739cfbe1ff208c1af3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:21:59.870817 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:21:59.872480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:21:59.886909 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccfef6-ab83-4891-bcc5-c2b850e32e53 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:21:59.928542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:21:59.945221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:21:59.945981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:21:59.946267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:21:59.946547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] PATH_INFO: `/v3/projects/804531b781f24296886ce74b67637d23` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:21:59.956892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=804531b781f24296886ce74b67637d23)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:21:59.958854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:22:00.010168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804531b781f24296886ce74b67637d23', 'request_id': 'req-6c8d5696-6363-4ec7-88c3-481fbfb895fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:22:00.015628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804531b781f24296886ce74b67637d23', 'request_id': 'req-6c8d5696-6363-4ec7-88c3-481fbfb895fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:22:00.024195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804531b781f24296886ce74b67637d23', 'request_id': 'req-6c8d5696-6363-4ec7-88c3-481fbfb895fb'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:22:00.036001 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] The token cache is being invalidate because project 804531b781f24296886ce74b67637d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:22:00.036293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:22:00.036911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:22:00.038463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1352/2683] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:21:59 2022] DELETE /identity/v3/projects/804531b781f24296886ce74b67637d23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:22:09.692287 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:22:10.032996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:22:51.050639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:22:51.051988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:22:51.124545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee621513-360f-4570-8edb-fd82ea12a23a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:22:51.187165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506b415-f604-468b-a093-fab40440d85f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:22:51.187978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b415-f604-468b-a093-fab40440d85f service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:22:51.188101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b415-f604-468b-a093-fab40440d85f service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:22:51.188365 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506b415-f604-468b-a093-fab40440d85f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:22:51.190284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5506b415-f604-468b-a093-fab40440d85f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:22:51.255946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b415-f604-468b-a093-fab40440d85f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:22:51.257513 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506b415-f604-468b-a093-fab40440d85f service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:22:51.275684 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1332/2684] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:22:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:23:01.273178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:23:41.139927 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:23:41.141349 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:23:41.206910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5696-6363-4ec7-88c3-481fbfb895fb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:23:41.210476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:23:41.211084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:23:41.211364 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:23:41.211639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:23:41.213342 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:23:41.271029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:23:41.272470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:23:41.281611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1353/2685] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:23:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:23:51.277494 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=52 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:41.376288 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506b415-f604-468b-a093-fab40440d85f service nova] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:41.377681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5506b415-f604-468b-a093-fab40440d85f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:41.389050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506b415-f604-468b-a093-fab40440d85f service nova] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:41.392625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:41.393234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:41.393518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:41.393796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:41.395987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:41.456600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:41.458563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32a58-4aed-4e1e-a570-e05b02df3d4b service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:41.468267 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1333/2686] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:24:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:24:41.530681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:41.532097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:41.595119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b756eb-ba2e-4706-9871-20b865df299e service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:41.600242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:41.600875 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:41.601161 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:41.601442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] PATH_INFO: `/v3/users/1f3343d49bbd4018899079222247a6e8` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:41.602641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3343d49bbd4018899079222247a6e8)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:41.604549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:41.650127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3343d49bbd4018899079222247a6e8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:41.650690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3343d49bbd4018899079222247a6e8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:41.658477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3343d49bbd4018899079222247a6e8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:41.671263 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3343d49bbd4018899079222247a6e8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:41.677996 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3343d49bbd4018899079222247a6e8'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:41.683751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1354/2687] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:24:41 2022] DELETE /identity/v3/users/1f3343d49bbd4018899079222247a6e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:24:41.688188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:41.688478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:41.688752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:41.708225 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:24:41.709410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:41.766022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1334/2688] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 13:24:41.776175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:41.777623 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:41.834229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7845e-f349-47ac-bc0b-e638eeb9507f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:41.837849 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:41.838488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:41.838775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:41.839056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:41.847978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:41.849432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:41.858902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1355/2689] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:24:42.131982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:42.133536 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:42.145604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127a7d7-c0db-40a4-82b3-efeeb6efb862 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:42.150482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:42.151095 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.151368 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.151740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] PATH_INFO: `/v3/projects/dae006e435104252b408c344ba1bab85` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.152868 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae006e435104252b408c344ba1bab85)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:42.154872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:42.210359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae006e435104252b408c344ba1bab85', 'request_id': 'req-274e5d7d-d884-4728-8416-a5f7c18142e4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.216678 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae006e435104252b408c344ba1bab85', 'request_id': 'req-274e5d7d-d884-4728-8416-a5f7c18142e4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.224576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae006e435104252b408c344ba1bab85', 'request_id': 'req-274e5d7d-d884-4728-8416-a5f7c18142e4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.234818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] The token cache is being invalidate because project dae006e435104252b408c344ba1bab85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:24:42.235099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.235710 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.237207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1335/2690] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:24:42 2022] DELETE /identity/v3/projects/dae006e435104252b408c344ba1bab85 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:24:42.242022 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:42.251965 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551310c7-529a-40bc-a5cd-1e1ad5eb164e service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:42.270813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:42.311597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:42.312386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.312673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.312950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] PATH_INFO: `/v3/users/093c0b58d5c74349af5a23155146517e` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.314148 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=093c0b58d5c74349af5a23155146517e)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:42.316089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:42.365691 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093c0b58d5c74349af5a23155146517e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.366402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093c0b58d5c74349af5a23155146517e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.374111 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093c0b58d5c74349af5a23155146517e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.386609 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093c0b58d5c74349af5a23155146517e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.393559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-2433d19f-3f6c-4a2d-983f-27ae73566586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093c0b58d5c74349af5a23155146517e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.399458 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1356/2691] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:24:42 2022] DELETE /identity/v3/users/093c0b58d5c74349af5a23155146517e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:24:42.666279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:42.667775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:42.685280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5d7d-d884-4728-8416-a5f7c18142e4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:42.698847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:42.745006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:42.745701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.745963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.746239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] PATH_INFO: `/v3/projects/fb7a1197a91f4926be74c4c06701b738` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.747385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb7a1197a91f4926be74c4c06701b738)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:42.749284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:42.794509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb7a1197a91f4926be74c4c06701b738', 'request_id': 'req-54a3b5be-21fd-46e4-915f-47eb14fef36d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.800466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb7a1197a91f4926be74c4c06701b738', 'request_id': 'req-54a3b5be-21fd-46e4-915f-47eb14fef36d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.808171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb7a1197a91f4926be74c4c06701b738', 'request_id': 'req-54a3b5be-21fd-46e4-915f-47eb14fef36d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.818377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] The token cache is being invalidate because project fb7a1197a91f4926be74c4c06701b738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:24:42.818665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.819284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-54a3b5be-21fd-46e4-915f-47eb14fef36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:24:42.820777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1336/2692] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:24:42 2022] DELETE /identity/v3/projects/fb7a1197a91f4926be74c4c06701b738 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:24:42.827141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.827945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.827945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.848574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:24:42.849770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:42.908211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1357/2693] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:24:42.916072 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.916361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.916636 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.936293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:24:42.937469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:42.945886 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1337/2694] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:24:42.949933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:42.959862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d9a30-d857-492d-87c0-dc59ea7fe84f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:42.964535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:42.965146 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.965432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.965708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:42.966463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:24:42.966768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:42.968347 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:42.979781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1358/2695] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 13:24:42.983840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:42.993566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856abcf-3268-4b34-b3aa-d71212af3bd8 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:42.998469 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:42.999003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:42.999283 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:42.999552 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.000214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.002175 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.028695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1338/2696] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:24:42 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:24:43.032811 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:43.042837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb31a6f-4b93-42b7-a8ee-4b8e1101c23a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:43.068539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:43.102271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:43.102891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.103174 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.103451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.104121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.106079 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.127428 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1359/2697] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:24:43 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:24:43.133264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:43.143483 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab6bb-4ba6-4c03-9894-f6a66bb7ddbb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:43.148517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:43.149134 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.149389 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.149659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.150377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.151949 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.162405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1339/2698] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:24:43.166374 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:43.177405 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57421f83-f48e-497c-af97-7ffb8aec2e0d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:43.183390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:43.184014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.184294 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.184572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.185272 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.186910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29268ba-6c47-45db-a850-70d6b051e6d8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.198590 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1360/2699] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:24:43.202806 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:43.213557 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa53e7-1496-488c-b2d1-3571acbc9713 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:43.218234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:43.218894 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.219178 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.219454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] PATH_INFO: `/v3/projects/db6efa95602746adb19945d6847fd562/users/69bfd1fed61a4f8bb70da86a48884dd2/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.233927 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6efa95602746adb19945d6847fd562, user_id=69bfd1fed61a4f8bb70da86a48884dd2, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.236568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.250794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1340/2700] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:24:43 2022] PUT /identity/v3/projects/db6efa95602746adb19945d6847fd562/users/69bfd1fed61a4f8bb70da86a48884dd2/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:24:43.261350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.261638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.261946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.290435 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] MFA Rules not processed for user `69bfd1fed61a4f8bb70da86a48884dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:24:43.291653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:43.357042 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1361/2701] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:24:43.368411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:43.370382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:43.421147 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:43.458633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ade3fe-dc54-47fe-9992-d9fd7bfe1c7b admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:43.462453 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:43.463100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:43.463382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:43.463658 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:43.472603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:43.474062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:43.483141 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1341/2702] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:24:45.096826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:24:45.099121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:45.170973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613adfd-ea6d-419e-92fe-416a5e163bfe None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:24:45.175902 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:24:45.176522 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:24:45.176804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:24:45.177080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:24:45.178871 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:24:45.190536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:24:45.192030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:24:45.201746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1362/2703] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:24:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:24:53.478832 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=15 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:24:55.197559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:26:49.713151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:26:49.714653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:26:49.788604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007a43-e959-4524-a76e-72d22df6a578 service glance] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:26:49.792264 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:26:49.792907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:26:49.793256 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:26:49.793532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:26:49.795478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:26:49.804909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:26:49.806372 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:26:49.814390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1342/2704] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:26:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:26:59.810795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=31 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:27:51.510632 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:27:51.512038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:27:51.579399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e689e1-5b15-4045-b80e-b34f0f7e58ba service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:27:51.582960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:27:51.583574 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:27:51.583854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:27:51.584131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:27:51.585846 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:27:51.643914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:27:51.645383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c21db6-92af-4e55-9b92-9908f08b37b6 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:27:51.654376 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1363/2705] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:01.650150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=53 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:44.107821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:44.188275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d7cb-90d0-4243-b82b-e6236d9d0b9e service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:44.193681 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:44.194380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.194666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.194942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] PATH_INFO: `/v3/users/69bfd1fed61a4f8bb70da86a48884dd2` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:44.196136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bfd1fed61a4f8bb70da86a48884dd2)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:44.198077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:44.246149 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bfd1fed61a4f8bb70da86a48884dd2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.253480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bfd1fed61a4f8bb70da86a48884dd2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.254136 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bfd1fed61a4f8bb70da86a48884dd2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.266018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bfd1fed61a4f8bb70da86a48884dd2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.266638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bfd1fed61a4f8bb70da86a48884dd2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.279479 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1343/2706] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:28:44 2022] DELETE /identity/v3/users/69bfd1fed61a4f8bb70da86a48884dd2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:28:44.284046 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.284513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.284696 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:44.323874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:44.325077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:44.383010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1364/2707] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:44.392501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:44.393946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:44.451177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d336-73da-44ec-b42b-96de58aee786 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:44.455410 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:44.456021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.456300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.456575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:44.465528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:44.466977 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb55ea9-9e23-4e2d-8ed4-29e76a067d7e service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:44.476353 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1344/2708] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:44.750587 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:44.752051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:44.764267 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5b3d6-c604-47f0-90d2-6f120c176ab4 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:44.769834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:44.770708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.770999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.771280 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] PATH_INFO: `/v3/projects/db6efa95602746adb19945d6847fd562` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:44.772472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6efa95602746adb19945d6847fd562)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:44.775277 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:44.825449 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6efa95602746adb19945d6847fd562', 'request_id': 'req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.831177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6efa95602746adb19945d6847fd562', 'request_id': 'req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.838660 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6efa95602746adb19945d6847fd562', 'request_id': 'req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.850741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] The token cache is being invalidate because project db6efa95602746adb19945d6847fd562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:28:44.851036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.851879 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-f7c51689-6b72-4317-bbe5-e59ea6ea11b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:44.853512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1365/2709] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:28:44 2022] DELETE /identity/v3/projects/db6efa95602746adb19945d6847fd562 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:28:44.858566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.858862 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.859143 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:44.865508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:44.909388 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:44.910531 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:44.967839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1345/2710] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:44.976035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:44.976321 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:44.976597 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.121560 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:45.122824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:45.131393 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1366/2711] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:45.135616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.145367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16ac3-b0a2-4d22-8e2c-14dcd2156d8f None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.150493 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.151097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.151381 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.151656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.152351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:28:45.152660 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.154276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.165579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1346/2712] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:28:45.169407 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.178951 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ee12a-0850-459a-9c8a-f838ad5e56fa None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.183487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.184074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.184350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.184616 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.185323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.187226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.222989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1367/2713] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:28:45 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:28:45.227022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.238049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c09ce5-7380-4744-8373-d70975618d4d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.297113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.297737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.298056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.298875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.299604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.301565 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.323323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1347/2714] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:28:45 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:28:45.327258 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.337452 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c708d80-684f-46f5-88a9-e2724af48794 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.342143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.342753 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.343049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.343309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.343999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.345726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.355695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1368/2715] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 13:28:45.359482 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.370109 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71a4bc-f432-4b73-b25f-3d7c2344242a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.375083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.375752 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.376037 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.376317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.377588 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.379248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164520-5c2f-4ee3-b4a2-7bb727c044fa admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.389772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1348/2716] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:28:45.393866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.403622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a22fa-91e9-401f-95ac-681c39c77dbf admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.408678 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.409327 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.409607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.409914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] PATH_INFO: `/v3/projects/171993bb813b4780b1e31c2c5133152a/users/3680a17bd1e04318a8538734367263ff/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.424759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] RBAC: Authorizing `identity:create_grant(project_id=171993bb813b4780b1e31c2c5133152a, user_id=3680a17bd1e04318a8538734367263ff, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.427352 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.429529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:45.441323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1369/2717] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:28:45 2022] PUT /identity/v3/projects/171993bb813b4780b1e31c2c5133152a/users/3680a17bd1e04318a8538734367263ff/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:28:45.452898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.453213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.453486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.480614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] MFA Rules not processed for user `3680a17bd1e04318a8538734367263ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:45.481797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:45.515534 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:45.549215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1349/2718] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:45.559778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:45.561508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:45.619002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56a9c-0b27-48cf-bca6-87f06a2a392f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:45.622998 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:45.623613 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:45.623957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:45.624256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:45.632662 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:45.634119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff3ae0-1c47-4edb-9eeb-d9fd84d4b3b6 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:45.642830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1370/2719] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:54.039564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.104420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0805-4721-41c7-aa9d-3530b53e3319 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:54.108947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:54.109580 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.109906 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.110174 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] PATH_INFO: `/v3/users/3680a17bd1e04318a8538734367263ff` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.111331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3680a17bd1e04318a8538734367263ff)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:54.113272 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:54.158995 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3680a17bd1e04318a8538734367263ff'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.167901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3680a17bd1e04318a8538734367263ff'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.168608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3680a17bd1e04318a8538734367263ff'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.180475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3680a17bd1e04318a8538734367263ff'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.181102 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3680a17bd1e04318a8538734367263ff'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.193566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1350/2720] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:28:54 2022] DELETE /identity/v3/users/3680a17bd1e04318a8538734367263ff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:28:54.198119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.198575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.198664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.217247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:54.218471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:54.279209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1371/2721] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:54.289535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.290985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:54.347685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8e3e2-a1a3-4fe5-bd4b-b054bcbee839 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:54.351346 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:54.351958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.352245 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.352518 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.361415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:54.362869 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23725741-0742-415f-bb8b-697cadab71ab service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:54.366784 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:54.373967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1351/2722] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:54.651189 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.652727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:54.665199 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774400-2cc8-43fc-b56b-7d516e6e9d65 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:54.670386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:54.671088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.671403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.671679 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] PATH_INFO: `/v3/projects/171993bb813b4780b1e31c2c5133152a` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.672830 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171993bb813b4780b1e31c2c5133152a)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:54.674763 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:54.724000 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171993bb813b4780b1e31c2c5133152a', 'request_id': 'req-243c183a-27ee-46a5-b329-30c69d262c64'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.729475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171993bb813b4780b1e31c2c5133152a', 'request_id': 'req-243c183a-27ee-46a5-b329-30c69d262c64'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.738308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171993bb813b4780b1e31c2c5133152a', 'request_id': 'req-243c183a-27ee-46a5-b329-30c69d262c64'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.749532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] The token cache is being invalidate because project 171993bb813b4780b1e31c2c5133152a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:28:54.749842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.750483 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-243c183a-27ee-46a5-b329-30c69d262c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:28:54.751993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1372/2723] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:28:54 2022] DELETE /identity/v3/projects/171993bb813b4780b1e31c2c5133152a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:28:54.757463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.757760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.758060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.777224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:54.778451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:54.849942 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1352/2724] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:54.859922 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.860279 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.860555 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.884823 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:54.886054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:54.893920 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1373/2725] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:54.898284 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.908056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082ff46-04b3-4e5d-9875-5bee15d39be2 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:54.912563 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:54.913165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.913447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.913722 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.914447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:28:54.914755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:54.916340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:54.928208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1353/2726] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:28:54.932250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.942084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5f768-0f74-48e3-998c-4067506c58ae None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:54.946704 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:54.947305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:54.947586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:54.947857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:54.948523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:54.950434 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:54.961408 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:54.978145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1374/2727] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:28:54 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:28:54.982214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:54.992564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb33608-c949-490b-9b8f-5d4714a44f23 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:55.047939 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:55.048533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.048820 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.049097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.050296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:55.052221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:55.072168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1354/2728] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:28:54 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:28:55.076253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:55.087126 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910746-a801-48bb-82ef-e59fb108638e admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:55.091890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:55.092512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.092793 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.093066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.093766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:55.095397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:55.106519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1375/2729] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 13:28:55.110506 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:55.121380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5239-719a-426e-977d-41a033c43c8f admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:55.126387 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:55.127000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.127278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.128169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.128865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:55.130497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df361e-41da-436e-80fa-dfd0c24e3a1c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:55.141755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1355/2730] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:28:55.146481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:55.156441 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0dafb-01b8-4be6-b581-00cfb3131512 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:55.161733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:55.162438 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.162724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.163004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] PATH_INFO: `/v3/projects/1c2c62fd3f254d4f89d36d71df10b659/users/e6fa32cc731743878ad4e45dba8d362d/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.177556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2c62fd3f254d4f89d36d71df10b659, user_id=e6fa32cc731743878ad4e45dba8d362d, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:55.180216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:55.193644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1376/2731] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:28:55 2022] PUT /identity/v3/projects/1c2c62fd3f254d4f89d36d71df10b659/users/e6fa32cc731743878ad4e45dba8d362d/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:28:55.198619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bcc47-d600-4463-b919-bd6df56328c1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.198935 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bcc47-d600-4463-b919-bd6df56328c1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.199208 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bcc47-d600-4463-b919-bd6df56328c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.218509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-804bcc47-d600-4463-b919-bd6df56328c1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:55.219697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-804bcc47-d600-4463-b919-bd6df56328c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:55.279454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1356/2732] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:55.290350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:55.292216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:55.348780 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda66f-60b6-4849-891d-3eabb9839a50 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:55.352397 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:55.353010 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:55.353295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:55.353578 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:55.362682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:55.364050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:55.373758 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1377/2733] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:28:56.000013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee57e4-84eb-4c43-b649-2b273c3f6cc1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:56.000307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee57e4-84eb-4c43-b649-2b273c3f6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:56.000595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee57e4-84eb-4c43-b649-2b273c3f6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:56.009422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:56.030211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d2ee57e4-84eb-4c43-b649-2b273c3f6cc1 None None] MFA Rules not processed for user `e6fa32cc731743878ad4e45dba8d362d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:56.031542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d2ee57e4-84eb-4c43-b649-2b273c3f6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:56.101034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1357/2734] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:56.113091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:56.114614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:56.180669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0a2ee-7a31-4e56-a224-1e5558e0c244 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:56.185933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:56.186565 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:56.186847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:56.187125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:56.197975 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:56.199495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:56.218415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1378/2735] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:57.576399 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:57.576891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:57.576980 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:57.596216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] MFA Rules not processed for user `e6fa32cc731743878ad4e45dba8d362d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:28:57.597419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:57.604358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1358/2736] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:28:57.644929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:57.646357 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:57.702227 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38402e-60ed-4b1c-a5ea-4026ef83ad75 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:57.705804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a9dc108d68404ba926c566516e7812', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:57.706974 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:57.707256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:57.707528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:57.709266 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:57.725673 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:28:57.797301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:57.798688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:57.819261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1379/2737] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:58.015896 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:58.017306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:58.029367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90680a0-d5a2-4bca-a4a4-5f96ba472436 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:58.032964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:58.033569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:58.033877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:58.034158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:58.035837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:58.045020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:58.046511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:58.055074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1359/2738] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:28:58.401734 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:58.403198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:58.410990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:28:58.419581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aec87-86d2-415b-8c29-d0782912ebfb service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:58.423756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:58.424358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:58.424643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:58.424946 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:58.426693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:58.440086 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:58.442070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:58.453491 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1360/2739] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:28:58.471608 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2438415-db2f-476f-b91e-97418292a19a service glance] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:28:58.476625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:28:58.477249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:28:58.477530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:28:58.477804 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:28:58.479641 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:28:58.491985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:28:58.494239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:28:58.505418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1380/2740] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:28:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:29:00.678494 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:00.680192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:00.698248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773c069-0ced-43c5-b574-394cd40b23d1 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:00.705113 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:00.705754 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:00.706125 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:00.706997 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:00.711235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:00.721861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:00.723460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc863bb-ea42-4fef-8d4a-305cd1edcd5d service nova] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:00.737860 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1361/2741] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:29:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:29:05.238709 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:05.254271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8144e7-0a97-406d-9d71-80ead7474929 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:05.260296 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a4c233f6ff1442b8e08fbe6400ecf0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:05.260917 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:05.261195 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:05.261472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:05.263269 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:05.276250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:05.277774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6e9e0-c060-4e47-97f7-77b19f8b6579 service nova] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:05.290094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1381/2742] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:29:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:29:09.588801 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:09.589247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:09.589361 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:09.609455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] MFA Rules not processed for user `e6fa32cc731743878ad4e45dba8d362d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:09.610723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:09.617648 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1362/2743] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:29:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:15.285409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=76 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:19.604535 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=54 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:27.071454 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e7555-8fb0-4966-bafc-890475c208c2 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:27.071935 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e7555-8fb0-4966-bafc-890475c208c2 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:27.072082 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e7555-8fb0-4966-bafc-890475c208c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:27.132261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-180e7555-8fb0-4966-bafc-890475c208c2 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:27.133413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-180e7555-8fb0-4966-bafc-890475c208c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:27.195557 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1382/2744] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 29 13:29:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:27.241342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:27.242805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:27.331910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b718c3-6833-4132-844c-da5ab5330c1e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:27.337790 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:27.338438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:27.338729 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:27.339005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:27.349638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:27.349731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:27.358000 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1363/2745] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 13:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 13:29:37.192683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=44 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:37.354787 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=41 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:42.834318 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:42.834776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:42.834866 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:42.863370 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] MFA Rules not processed for user `c26a3ad083a544e996ac6e4dcdc12a37`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:42.864566 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:42.882316 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1383/2746] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 29 13:29:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:42.896325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:42.915948 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc655f-e765-43ba-b072-d36d3e4923d3 service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:42.920971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3dffd8d4deb4cadae3402ffc79ad123', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:42.921604 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:42.921921 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:42.922198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:42.931097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:42.932643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:42.941313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1364/2747] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 29 13:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 13:29:44.776910 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:44.798615 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296632b-3ede-4e08-ba46-ff7098a4b235 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:44.804947 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:44.805579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:44.805894 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:44.806177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:44.808089 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:44.837414 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:44.839303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:44.841033 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:44.850742 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446544-c46f-4b59-ba7f-9ef3a8337845 service swift] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:44.852994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1384/2748] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:29:44.859677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:44.860558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:44.860840 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:44.865133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] PATH_INFO: `/v3/users/e6fa32cc731743878ad4e45dba8d362d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:44.866444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fa32cc731743878ad4e45dba8d362d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:44.868558 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:44.918335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fa32cc731743878ad4e45dba8d362d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:44.926107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fa32cc731743878ad4e45dba8d362d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:44.926783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fa32cc731743878ad4e45dba8d362d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:44.938570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fa32cc731743878ad4e45dba8d362d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:44.939224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-899cb2a8-e1c4-4e0c-9240-4f98d459c13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fa32cc731743878ad4e45dba8d362d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:44.951655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1365/2749] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:29:44 2022] DELETE /identity/v3/users/e6fa32cc731743878ad4e45dba8d362d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:29:45.130964 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.144323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7f2ce-7a5b-49e8-8b5b-7b000e94b582 service placement] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.199518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.200155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.200439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.200712 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] PATH_INFO: `/v3/projects/1c2c62fd3f254d4f89d36d71df10b659` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.201885 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2c62fd3f254d4f89d36d71df10b659)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.203768 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.249386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2c62fd3f254d4f89d36d71df10b659', 'request_id': 'req-64aac88f-c3d9-4588-b706-b10933ba45cc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:45.255336 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2c62fd3f254d4f89d36d71df10b659', 'request_id': 'req-64aac88f-c3d9-4588-b706-b10933ba45cc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:45.262150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2c62fd3f254d4f89d36d71df10b659', 'request_id': 'req-64aac88f-c3d9-4588-b706-b10933ba45cc'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:45.273570 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] The token cache is being invalidate because project 1c2c62fd3f254d4f89d36d71df10b659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:29:45.273888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:45.274589 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-64aac88f-c3d9-4588-b706-b10933ba45cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:29:45.276219 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1385/2750] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:29:45 2022] DELETE /identity/v3/projects/1c2c62fd3f254d4f89d36d71df10b659 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:29:45.281061 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.281348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.281621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.302238 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:45.303852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:45.366100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1366/2751] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:45.374499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.374789 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.375065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.395415 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:45.396554 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:45.404338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1386/2752] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:45.408408 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.418510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f44f3-53b0-4dcf-b45a-54a6cebabc81 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.423243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.423898 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.424184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.424457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.425155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:29:45.425462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.427088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.438633 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1367/2753] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:29:45.443507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.452705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812439fc-cc6f-4a98-b1bc-c238d86a4803 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.457976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.458581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.458861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.459138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.459795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.461664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.488842 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1387/2754] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:29:45 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:29:45.492842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.503838 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f969a4-d2ac-4c49-afd0-6fa3ff976f88 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.541380 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:45.561266 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.561910 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.562199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.562476 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.563128 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.565548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.587279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1368/2755] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:29:45 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:29:45.591359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.598047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:45.602375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c1c44-2919-40c9-869a-8e77413a3627 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.608121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.608928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.609423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.609903 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.610626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.612213 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.622322 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1388/2756] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:29:45.626139 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.636405 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624fa-0128-4838-8188-78842fb7d9c1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.640999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.641597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.641907 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.642183 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.643550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.645119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b8235-fcf2-4978-9093-106b3f4a948e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.654911 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1369/2757] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:29:45.658644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.668304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c898a-5820-4999-ace1-d9a88895cc82 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.672760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.673403 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.673681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.673985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] PATH_INFO: `/v3/projects/6592f03a6ecb486e9553dc463a5d0d56/users/740c66e7c3ce4f43a1bd26ea9edb2411/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.687914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6592f03a6ecb486e9553dc463a5d0d56, user_id=740c66e7c3ce4f43a1bd26ea9edb2411, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.691055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.703934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1389/2758] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:29:45 2022] PUT /identity/v3/projects/6592f03a6ecb486e9553dc463a5d0d56/users/740c66e7c3ce4f43a1bd26ea9edb2411/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:29:45.714204 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.714496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.714774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.742511 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] MFA Rules not processed for user `740c66e7c3ce4f43a1bd26ea9edb2411`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:29:45.743715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:45.808455 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1370/2759] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:29:45.820471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:45.822249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:45.878880 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be973a1-0051-4f3c-acc9-a44541c41d61 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:45.882540 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:45.883154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:45.883432 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:45.883708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:45.891904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:45.893345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:45.902075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1390/2760] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:29:46.948768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:29:46.950241 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:47.018320 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff78345-b163-4958-a3cb-ad14b3042c08 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:29:47.023679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:29:47.024289 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:29:47.024569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:29:47.024839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:29:47.026571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:29:47.037897 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:29:47.039460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:29:47.049259 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1371/2761] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:29:55.898095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:29:57.044965 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:03.135124 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:03.136538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:03.194383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908dc32-b61b-4492-97c5-f74ebdbdf4ff service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:03.197999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:03.198621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:03.198904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:03.199211 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:03.200972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:03.264074 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:03.265507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:03.275106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1391/2762] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:06.621443 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:06.623671 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:06.637591 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1c3e7-7352-484e-8788-43c640fcf2bf service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:06.642168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:06.642760 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:06.643045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:06.643812 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] PATH_INFO: `/v3/users/03f312d5c7334ccc82be17c5dfe29aeb` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:06.658044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f312d5c7334ccc82be17c5dfe29aeb)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:06.660669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:06.705261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f312d5c7334ccc82be17c5dfe29aeb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:06.712639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f312d5c7334ccc82be17c5dfe29aeb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:06.713295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f312d5c7334ccc82be17c5dfe29aeb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:06.726891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f312d5c7334ccc82be17c5dfe29aeb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:06.727606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-abcf8846-8921-4a04-8f6e-f8e1f081f68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f312d5c7334ccc82be17c5dfe29aeb'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:06.740968 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1372/2763] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:06 2022] DELETE /identity/v3/users/03f312d5c7334ccc82be17c5dfe29aeb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:30:06.920152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:06.936305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65cfe6-be10-4ab3-a1cc-4862cec5491e service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:06.996368 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:06.996938 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:06.997223 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:06.997503 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] PATH_INFO: `/v3/projects/f36dffd2ab1643ed939ffe87e5928187` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:06.998713 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36dffd2ab1643ed939ffe87e5928187)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.000602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.047724 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36dffd2ab1643ed939ffe87e5928187', 'request_id': 'req-ed8b977a-61e8-496a-a032-52fd88853851'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:07.050514 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:07.055293 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36dffd2ab1643ed939ffe87e5928187', 'request_id': 'req-ed8b977a-61e8-496a-a032-52fd88853851'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:07.062386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36dffd2ab1643ed939ffe87e5928187', 'request_id': 'req-ed8b977a-61e8-496a-a032-52fd88853851'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:07.073750 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] The token cache is being invalidate because project f36dffd2ab1643ed939ffe87e5928187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:07.074073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:07.074766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ed8b977a-61e8-496a-a032-52fd88853851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:07.076412 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1392/2764] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:06 2022] DELETE /identity/v3/projects/f36dffd2ab1643ed939ffe87e5928187 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:07.082318 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.082614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.082909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.105727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:07.107151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:07.172523 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1373/2765] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:07.180619 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.180905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.181181 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.200505 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:07.201722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:07.210542 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1393/2766] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:07.214821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.225240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e19cd-d281-4e51-89c8-0dff4414bd11 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.230087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.230698 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.230984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.231262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.231961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:07.232265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.233892 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.245181 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1374/2767] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:07.249966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.260051 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6bfff-84bd-4523-b0d7-3bfc0011b2c0 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.270383 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.271009 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.271295 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.271573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.272253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.274179 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.302303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1394/2768] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:07 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:07.306466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.316883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff163-11ab-45b3-b316-a69d2592da47 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.363510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:07.377092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.377716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.378034 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.378311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.378970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.380893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.402441 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1375/2769] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:07 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:07.406759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.417748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135109d-1eb0-40c7-88fe-3b0757fc486b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.422448 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.423063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.423345 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.423622 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.424409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.426264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.437169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1395/2770] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:07.441191 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.452281 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49bae-f9d8-4b2f-a98d-b479101e18e7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.457852 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.458510 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.458789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.459063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.459764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.461383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580f8a2-dd76-41a3-a6a9-75da5e78d437 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.473475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1376/2771] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:07.477710 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.488802 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee241-f503-4d6b-a6a9-b17439021c46 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.493528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.494231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.494518 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.494798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] PATH_INFO: `/v3/projects/d112891f7fac401ab55dd9388668973a/users/fc42ae2ef25f4b529d5f6563093c9f52/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.512314 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d112891f7fac401ab55dd9388668973a, user_id=fc42ae2ef25f4b529d5f6563093c9f52, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.515011 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.533869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1396/2772] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:07 2022] PUT /identity/v3/projects/d112891f7fac401ab55dd9388668973a/users/fc42ae2ef25f4b529d5f6563093c9f52/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:30:07.535638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7a200-af75-4250-8b39-a4491565d1b1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.535999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7a200-af75-4250-8b39-a4491565d1b1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.536303 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7a200-af75-4250-8b39-a4491565d1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.555990 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-31b7a200-af75-4250-8b39-a4491565d1b1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:07.557206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-31b7a200-af75-4250-8b39-a4491565d1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:07.618861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1377/2773] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:07.629099 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:07.630877 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:07.693077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c103d-91f0-450b-8a45-011faa809bd7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:07.696725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915c145-7236-4403-863c-684a9606349f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:07.697344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915c145-7236-4403-863c-684a9606349f service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:07.697626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915c145-7236-4403-863c-684a9606349f service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:07.697933 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915c145-7236-4403-863c-684a9606349f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:07.707451 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915c145-7236-4403-863c-684a9606349f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:07.708864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915c145-7236-4403-863c-684a9606349f service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:07.718209 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1397/2774] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:08.373074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.373074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.373074 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.400462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] MFA Rules not processed for user `fc42ae2ef25f4b529d5f6563093c9f52`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:08.401675 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:08.478120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1378/2775] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:08.482800 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915c145-7236-4403-863c-684a9606349f service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.494114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915c145-7236-4403-863c-684a9606349f service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.499064 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.499746 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.500108 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.500371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.501041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.503057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.536633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1398/2776] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:08 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:08.537938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.548943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58ab49-9bea-4b46-8d33-9b790272f31e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.602620 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:08.606390 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.607263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.607581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.611629 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.612917 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.614940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.637568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1379/2777] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:08 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:08.642507 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.653668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa892-86c5-464f-9cc3-64538b996c4c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.658401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.659727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.660002 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.660273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.660966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.662602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.674268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1399/2778] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:08.678165 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.688358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016b22-7682-4b32-a6af-6cbd653ba757 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.693937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.694615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.694901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.695279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] PATH_INFO: `/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db/users/a1bf4a33cde34c86b917da7037d6fe95/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.710846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c822ea0e3472690c4bd78cbd7c9db, user_id=a1bf4a33cde34c86b917da7037d6fe95, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.713475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.727628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1380/2779] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:08 2022] PUT /identity/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db/users/a1bf4a33cde34c86b917da7037d6fe95/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:08.731627 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.741601 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c88be-a3e4-4827-b33e-6d945af88f40 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.788048 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:08.802476 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.803076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.803359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.803639 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.804330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.805986 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdf011-3747-4bd3-b418-5cff95690d27 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.816112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1400/2780] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:08.820475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:08.831571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbe70d-8cdc-4730-bf74-1f5fb36caad1 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:08.836548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:08.837481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:08.837762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:08.838082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] PATH_INFO: `/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db/users/a1bf4a33cde34c86b917da7037d6fe95/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:08.840229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6c822ea0e3472690c4bd78cbd7c9db, user_id=a1bf4a33cde34c86b917da7037d6fe95, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:08.842984 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:08.854067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1381/2781] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:08 2022] PUT /identity/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db/users/a1bf4a33cde34c86b917da7037d6fe95/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:09.510551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:09.511073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:09.511186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:09.543104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] MFA Rules not processed for user `a1bf4a33cde34c86b917da7037d6fe95`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:09.544324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:09.629057 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1401/2782] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:10.130994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:10.132805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:10.212866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bedbc0-7bb0-42d8-a7fc-3553a11129ee admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:10.218576 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:10.219299 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:10.219656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:10.220005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:10.222049 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:10.236223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:10.238463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:10.249278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1382/2783] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:10.723942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:10.798095 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e14312-3410-4879-a47b-ac9bf4637583 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:10.803330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:10.803976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:10.804255 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:10.804530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] PATH_INFO: `/v3/users/fc42ae2ef25f4b529d5f6563093c9f52` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:10.805721 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc42ae2ef25f4b529d5f6563093c9f52)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:10.807686 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:10.852890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc42ae2ef25f4b529d5f6563093c9f52'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:10.853539 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc42ae2ef25f4b529d5f6563093c9f52'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:10.860914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc42ae2ef25f4b529d5f6563093c9f52'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:10.872934 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:10.874178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc42ae2ef25f4b529d5f6563093c9f52'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:10.882867 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc42ae2ef25f4b529d5f6563093c9f52'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:10.889041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1402/2784] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:10 2022] DELETE /identity/v3/users/fc42ae2ef25f4b529d5f6563093c9f52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:11.063955 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.085079 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a68337-00d0-4075-8652-00c2c38b1c82 service placement] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:11.143420 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:11.144076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.144331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.144606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] PATH_INFO: `/v3/projects/d112891f7fac401ab55dd9388668973a` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.145762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d112891f7fac401ab55dd9388668973a)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:11.147693 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:11.188610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:11.195877 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd112891f7fac401ab55dd9388668973a', 'request_id': 'req-b7970ddf-b524-4371-9352-3ae56b9b9093'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.202326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd112891f7fac401ab55dd9388668973a', 'request_id': 'req-b7970ddf-b524-4371-9352-3ae56b9b9093'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.209560 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd112891f7fac401ab55dd9388668973a', 'request_id': 'req-b7970ddf-b524-4371-9352-3ae56b9b9093'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.219704 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] The token cache is being invalidate because project d112891f7fac401ab55dd9388668973a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:11.219987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.220611 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.222169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1383/2785] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:11 2022] DELETE /identity/v3/projects/d112891f7fac401ab55dd9388668973a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:30:11.226087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.227487 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:11.241230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4380d-3fa2-4913-9e49-5b4ab1995e53 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:11.296437 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:11.297071 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.297350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.297630 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] PATH_INFO: `/v3/users/a1bf4a33cde34c86b917da7037d6fe95` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.298807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bf4a33cde34c86b917da7037d6fe95)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:11.301304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:11.350568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bf4a33cde34c86b917da7037d6fe95'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.351226 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bf4a33cde34c86b917da7037d6fe95'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.358984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bf4a33cde34c86b917da7037d6fe95'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.372951 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bf4a33cde34c86b917da7037d6fe95'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.379808 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e52b0565-6fd0-43b2-927b-9815ce6c4261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bf4a33cde34c86b917da7037d6fe95'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.385680 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1403/2786] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:11 2022] DELETE /identity/v3/users/a1bf4a33cde34c86b917da7037d6fe95 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:11.569314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.570835 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:11.587311 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7970ddf-b524-4371-9352-3ae56b9b9093 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:11.648821 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:11.649468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.649753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.650064 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] PATH_INFO: `/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.651222 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6c822ea0e3472690c4bd78cbd7c9db)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:11.653161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:11.698068 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6c822ea0e3472690c4bd78cbd7c9db', 'request_id': 'req-71b3ce24-8b57-4846-a4fc-e670360d2071'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.704005 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6c822ea0e3472690c4bd78cbd7c9db', 'request_id': 'req-71b3ce24-8b57-4846-a4fc-e670360d2071'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.715421 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6c822ea0e3472690c4bd78cbd7c9db', 'request_id': 'req-71b3ce24-8b57-4846-a4fc-e670360d2071'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.729041 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] The token cache is being invalidate because project 4b6c822ea0e3472690c4bd78cbd7c9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:11.729344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.730029 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-71b3ce24-8b57-4846-a4fc-e670360d2071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:11.734845 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1384/2787] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:11 2022] DELETE /identity/v3/projects/4b6c822ea0e3472690c4bd78cbd7c9db => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:11.737553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.737553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.737553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.757338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:11.758697 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:11.816285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1404/2788] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:11.824202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.824437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.824716 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.844837 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:11.846080 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:11.854944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1385/2789] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:11.859186 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.868792 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc7dfe-5f5d-44a9-a9d8-0d8bfc80c06f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:11.873681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:11.874344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.874628 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.874905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.875604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:11.875915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:11.877519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:11.879125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:11.891156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1405/2790] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:11.895551 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.904715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62534c59-93ae-494c-b873-b7c91f128fb1 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:11.910052 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:11.910656 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:11.910940 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:11.911216 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:11.911882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:11.913764 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:11.941066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1386/2791] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:11 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:11.945003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:11.954809 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1e636-17cf-4e96-b5c0-60dccfcf8ad3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.010911 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.011536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.011805 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.012103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.012762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.014695 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.036596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1406/2792] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:11 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:12.044540 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.055808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de054a9a-4b31-41bc-b4b5-e765101f9a32 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.060435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.061042 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.061325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.061600 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.062398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.064016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.074498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1387/2793] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:12.078814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.090285 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423250-80b9-4e96-89c7-d764b86c660a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.094973 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.095580 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.095864 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.096165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.096862 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.098495 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee6245-889c-414d-92ec-00e06d79fd4e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.109166 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1407/2794] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:12.113138 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.123637 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57e8c0-ebfd-4df4-a64a-ce9615cc9fea admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.128923 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.129619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.130014 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.130333 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] PATH_INFO: `/v3/projects/59867d98ab7a421fa7d49e6a5b0fca08/users/c6a9689a7a2c460bb27fe410fb4c6ab9/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.146152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59867d98ab7a421fa7d49e6a5b0fca08, user_id=c6a9689a7a2c460bb27fe410fb4c6ab9, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.148781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.151805 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:12.164099 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1388/2795] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:12 2022] PUT /identity/v3/projects/59867d98ab7a421fa7d49e6a5b0fca08/users/c6a9689a7a2c460bb27fe410fb4c6ab9/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:30:12.174198 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.174510 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.174788 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.201873 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] MFA Rules not processed for user `c6a9689a7a2c460bb27fe410fb4c6ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:12.203065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:12.267914 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1408/2796] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:12.272973 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.333797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780b17a-005e-478d-b8f3-91c59b0f9776 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.338499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.339115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.339401 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.339702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.340370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.342402 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.368727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1389/2797] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:12 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:12.373083 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.384248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963059-e04c-4120-b4bd-934347cb5217 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.439681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.440908 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.441191 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.441467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.442167 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.444076 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.452738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:12.464593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1409/2798] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:12 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:12.468475 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.478777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d978de9-9796-4617-83b0-1c4e1d1028a7 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.483467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.484066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.484348 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.484623 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.485313 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.487449 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.498467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1390/2799] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:12.503173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.512945 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34606bb2-07ba-4e78-b442-4e743512d06b admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.517463 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.518157 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.518439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.518715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] PATH_INFO: `/v3/projects/ae891f9590414d359461b8f701b81337/users/28531c4321dc4992846c676930b6325d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.533401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae891f9590414d359461b8f701b81337, user_id=28531c4321dc4992846c676930b6325d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.536264 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.548905 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1410/2800] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:12 2022] PUT /identity/v3/projects/ae891f9590414d359461b8f701b81337/users/28531c4321dc4992846c676930b6325d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:12.553522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.563533 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144037-073b-4e0d-9212-15119033f142 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.620077 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.620616 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.620909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.621436 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.622160 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.623753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aea24-bcb9-4ef8-9438-50e6044c374f admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.633936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1391/2801] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:12.638418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.648607 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a9b16-bb7b-459d-866c-2ac73a1a62d4 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.653169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.653847 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.654143 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.654423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] PATH_INFO: `/v3/projects/ae891f9590414d359461b8f701b81337/users/28531c4321dc4992846c676930b6325d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.656402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae891f9590414d359461b8f701b81337, user_id=28531c4321dc4992846c676930b6325d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.659043 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.668978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1411/2802] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:12 2022] PUT /identity/v3/projects/ae891f9590414d359461b8f701b81337/users/28531c4321dc4992846c676930b6325d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:12.680879 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.681171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.681451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.709007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] MFA Rules not processed for user `28531c4321dc4992846c676930b6325d`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:12.710240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:12.760744 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:12.778718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1392/2803] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:12.789972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:12.791681 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:12.847844 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911cdd69-a614-44f3-a0ff-e39692c45cfb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:12.851486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:12.852094 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:12.852378 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:12.852656 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:12.908419 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:12.910386 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:12.919388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1412/2804] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:18.563101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:18.572795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595bf60-1743-4b46-96c4-09984bc0844c None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:18.576326 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:18.576936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:18.577214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:18.577489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:18.587206 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:18.588627 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd0d56-e910-42ef-bdd8-e582535919ab service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:18.598548 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1393/2805] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:22.915301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:28.593756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=10 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:30.556362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:30.557766 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:30.622621 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab695a-c5e7-4c18-90c2-990d6b7e768b service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:30.627104 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:30.627743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:30.628032 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:30.628310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] PATH_INFO: `/v3/users/740c66e7c3ce4f43a1bd26ea9edb2411` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:30.629513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740c66e7c3ce4f43a1bd26ea9edb2411)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:30.632030 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:30.677959 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740c66e7c3ce4f43a1bd26ea9edb2411'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:30.678638 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740c66e7c3ce4f43a1bd26ea9edb2411'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:30.685863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740c66e7c3ce4f43a1bd26ea9edb2411'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:30.698323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740c66e7c3ce4f43a1bd26ea9edb2411'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:30.705091 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740c66e7c3ce4f43a1bd26ea9edb2411'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:30.711470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1413/2806] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:30 2022] DELETE /identity/v3/users/740c66e7c3ce4f43a1bd26ea9edb2411 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:30.716070 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:30.716528 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:30.716624 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:30.736777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:30.737982 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:30.799643 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1394/2807] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:30.809760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:30.811237 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:30.866698 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afbe8-3dc5-4df6-8b9b-16dc9a4555b7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:30.870328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:30.870936 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:30.871216 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:30.871749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:30.880532 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:30.882547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56b7f1-dbca-4060-8b75-a6363c4502a3 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:30.891716 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1414/2808] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:31.158195 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.159621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:31.172060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14a419-63ae-42e5-85c0-b0178583d9a3 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.176639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.177279 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.177559 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.178501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] PATH_INFO: `/v3/projects/6592f03a6ecb486e9553dc463a5d0d56` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.179667 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6592f03a6ecb486e9553dc463a5d0d56)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.181579 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.228082 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6592f03a6ecb486e9553dc463a5d0d56', 'request_id': 'req-690ea189-cd4b-49cc-bfd2-12a6334df1c5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:31.234330 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6592f03a6ecb486e9553dc463a5d0d56', 'request_id': 'req-690ea189-cd4b-49cc-bfd2-12a6334df1c5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:31.244415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6592f03a6ecb486e9553dc463a5d0d56', 'request_id': 'req-690ea189-cd4b-49cc-bfd2-12a6334df1c5'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:31.257641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] The token cache is being invalidate because project 6592f03a6ecb486e9553dc463a5d0d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:31.257972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:31.258610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-690ea189-cd4b-49cc-bfd2-12a6334df1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:31.260122 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1395/2809] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:31 2022] DELETE /identity/v3/projects/6592f03a6ecb486e9553dc463a5d0d56 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:31.265017 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.265308 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.265585 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.284581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:31.285775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:31.303614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:31.346333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1415/2810] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:31.354451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.354743 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.355022 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.375335 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:31.376561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:31.385087 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1396/2811] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:31.389145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.398682 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af98803-a8f9-4136-b1f7-c21f2a37af6e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.403210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98a53-406a-4165-882d-3866667916da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.403819 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98a53-406a-4165-882d-3866667916da admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.404102 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98a53-406a-4165-882d-3866667916da admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.404377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98a53-406a-4165-882d-3866667916da admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.405066 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98a53-406a-4165-882d-3866667916da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:31.405372 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98a53-406a-4165-882d-3866667916da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.406983 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98a53-406a-4165-882d-3866667916da admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.418683 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1416/2812] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:31.422763 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.432592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf9d61-908a-47fc-beb3-fbab03417045 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.437282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.437918 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.438297 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.438467 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.439188 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.440988 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.468101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1397/2813] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:31 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:31.472242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98a53-406a-4165-882d-3866667916da admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.482854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98a53-406a-4165-882d-3866667916da admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.537895 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.538508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.538791 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.539063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.539711 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.541603 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.563035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1417/2814] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:31 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:31.566978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.577345 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc298a-2ddb-440c-98ec-cd42169ca0d6 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.581972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.582575 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.582858 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.583135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.583831 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.585986 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.595775 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1398/2815] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:31.599536 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.609705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58c73-b8bf-40a1-bdf9-fa40467bced9 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.614330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.614928 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.615210 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.615486 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.616173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.617738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486aee-09e5-4912-a1a8-8a5ecf8eb5e0 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.629160 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1418/2816] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:31.633100 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.643091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ead45-832d-43b0-8bb7-db8f88db9567 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.648358 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.649008 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.649293 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.649569 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] PATH_INFO: `/v3/projects/9a86813dfa9c4d0d8c0217d29d1318f5/users/96422547f7fb45a7a0920e520d66870c/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.665059 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a86813dfa9c4d0d8c0217d29d1318f5, user_id=96422547f7fb45a7a0920e520d66870c, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.667727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.680603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1399/2817] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:31 2022] PUT /identity/v3/projects/9a86813dfa9c4d0d8c0217d29d1318f5/users/96422547f7fb45a7a0920e520d66870c/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:31.685529 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13792620-5ee3-4f2b-9383-6ec5dc7c3a56 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.685838 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13792620-5ee3-4f2b-9383-6ec5dc7c3a56 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.686119 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13792620-5ee3-4f2b-9383-6ec5dc7c3a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.704329 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-13792620-5ee3-4f2b-9383-6ec5dc7c3a56 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:31.705519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-13792620-5ee3-4f2b-9383-6ec5dc7c3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:31.762121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1419/2818] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 13:30:31.772570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:31.774278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:31.796438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:31.833419 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d35-af8b-499c-9f86-0e55bd944272 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:31.837614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:31.838322 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:31.838603 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:31.838875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:31.849462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:31.851013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:31.860278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1400/2819] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:32.482274 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ef77-176e-46c8-ba67-7faf07407db0 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:32.482583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ef77-176e-46c8-ba67-7faf07407db0 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:32.482865 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ef77-176e-46c8-ba67-7faf07407db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:32.511474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-de60ef77-176e-46c8-ba67-7faf07407db0 None None] MFA Rules not processed for user `96422547f7fb45a7a0920e520d66870c`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:32.512688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-de60ef77-176e-46c8-ba67-7faf07407db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:32.531149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:32.585782 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1420/2820] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:32.596515 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:32.597958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:32.655771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53f575-71f7-4e01-bfd9-50b6c10aedf8 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:32.659477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:32.660197 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:32.660374 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:32.660650 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:32.669699 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:32.671107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:32.680841 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1401/2821] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:34.033708 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e46825-f5ec-4bac-8fd4-a67ff7cb6e03 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:34.034230 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e46825-f5ec-4bac-8fd4-a67ff7cb6e03 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:34.034338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e46825-f5ec-4bac-8fd4-a67ff7cb6e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:34.053591 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-c6e46825-f5ec-4bac-8fd4-a67ff7cb6e03 None None] MFA Rules not processed for user `96422547f7fb45a7a0920e520d66870c`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:34.054839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c6e46825-f5ec-4bac-8fd4-a67ff7cb6e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:34.061689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1421/2822] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:42.452985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:42.454481 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:42.468198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ec981-40ba-4876-b712-0c54f0ac43bb service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:42.472933 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:42.473599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:42.473936 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:42.474211 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] PATH_INFO: `/v3/users/c6a9689a7a2c460bb27fe410fb4c6ab9` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:42.475881 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a9689a7a2c460bb27fe410fb4c6ab9)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:42.477874 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:42.525685 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a9689a7a2c460bb27fe410fb4c6ab9'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:42.533019 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a9689a7a2c460bb27fe410fb4c6ab9'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:42.533861 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a9689a7a2c460bb27fe410fb4c6ab9'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:42.546597 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a9689a7a2c460bb27fe410fb4c6ab9'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:42.547646 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a9689a7a2c460bb27fe410fb4c6ab9'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:42.562797 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1402/2823] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:42 2022] DELETE /identity/v3/users/c6a9689a7a2c460bb27fe410fb4c6ab9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:42.567479 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:42.567989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:42.567989 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:42.587180 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:42.588373 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:42.649535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1422/2824] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:42.660699 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:42.662659 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:42.705944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:42.723987 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489913-cc5a-4bca-a4e2-f1e5e1a3e5ff admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:42.728383 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:42.729058 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:42.729342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:42.729621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:42.739215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:42.740695 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:42.750641 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1403/2825] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:43.086221 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:43.087727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:43.101573 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb853cac-64ab-4352-a74b-3011661c769a None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:43.106141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:43.106781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.107577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.107856 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] PATH_INFO: `/v3/projects/59867d98ab7a421fa7d49e6a5b0fca08` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.109208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=59867d98ab7a421fa7d49e6a5b0fca08)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:43.111290 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:43.158887 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59867d98ab7a421fa7d49e6a5b0fca08', 'request_id': 'req-9f31668f-98d5-4c31-b5b5-eb064fe11cac'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.164801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59867d98ab7a421fa7d49e6a5b0fca08', 'request_id': 'req-9f31668f-98d5-4c31-b5b5-eb064fe11cac'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.171525 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59867d98ab7a421fa7d49e6a5b0fca08', 'request_id': 'req-9f31668f-98d5-4c31-b5b5-eb064fe11cac'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.182667 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] The token cache is being invalidate because project 59867d98ab7a421fa7d49e6a5b0fca08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:43.182958 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.183584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.185131 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1423/2826] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:43 2022] DELETE /identity/v3/projects/59867d98ab7a421fa7d49e6a5b0fca08 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:43.189062 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:43.199196 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1384-86a6-4d7a-b559-705721f205ee service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:43.259758 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:43.260403 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.260686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.260969 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] PATH_INFO: `/v3/users/28531c4321dc4992846c676930b6325d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.262153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28531c4321dc4992846c676930b6325d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:43.264228 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:43.310994 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28531c4321dc4992846c676930b6325d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.319124 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28531c4321dc4992846c676930b6325d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.319781 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28531c4321dc4992846c676930b6325d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.331584 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28531c4321dc4992846c676930b6325d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.332247 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-3530e2a8-8cf8-45ee-9a13-785d91028da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28531c4321dc4992846c676930b6325d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.348961 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1404/2827] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:43 2022] DELETE /identity/v3/users/28531c4321dc4992846c676930b6325d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:43.634971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:43.636436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:43.651310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31668f-98d5-4c31-b5b5-eb064fe11cac admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:43.686556 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:43.712749 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:43.713323 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.713602 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.713904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] PATH_INFO: `/v3/projects/ae891f9590414d359461b8f701b81337` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.715061 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae891f9590414d359461b8f701b81337)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:43.716942 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:43.762006 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae891f9590414d359461b8f701b81337', 'request_id': 'req-bdff9552-a91c-49a8-92a2-a7fc31a0266a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.767399 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae891f9590414d359461b8f701b81337', 'request_id': 'req-bdff9552-a91c-49a8-92a2-a7fc31a0266a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.774259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae891f9590414d359461b8f701b81337', 'request_id': 'req-bdff9552-a91c-49a8-92a2-a7fc31a0266a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.785072 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] The token cache is being invalidate because project ae891f9590414d359461b8f701b81337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:43.785363 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.786055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bdff9552-a91c-49a8-92a2-a7fc31a0266a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:43.787546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1424/2828] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:43 2022] DELETE /identity/v3/projects/ae891f9590414d359461b8f701b81337 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:43.792489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.792937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.793046 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.813771 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:43.815003 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:43.865647 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:43.875398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1405/2829] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:43.883338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.883643 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.883929 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.904722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:43.905987 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:43.914815 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1425/2830] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:43.919224 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:43.929718 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada4201-114a-48cb-945b-3fa3e28ccd55 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:43.934444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:43.935056 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.935340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.935614 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.936310 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:43.936615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:43.938239 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:43.953478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1406/2831] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:43.955423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:43.967500 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86994c4d-6359-4d13-9396-0a3bb673a2ac None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:43.972527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:43.973145 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:43.973425 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:43.973972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:43.975137 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:43.977103 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.006889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1426/2832] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:43 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:44.011012 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.020947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca188-a8c3-4b2f-8818-6a3e57315f24 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.079653 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.080338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.080677 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.081030 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.081783 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.083888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.104888 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1407/2833] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:44 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:44.108972 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.120202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd51ecd-e370-4c32-8152-050f58d830e1 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.125303 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.125978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.126282 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.126586 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.127283 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.128855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.138857 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1427/2834] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:44.142756 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.152458 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893ad6c-1b1c-478d-8f09-92db462414db admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.156950 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.157543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.157902 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.158114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.158794 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.160356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e632eb-1b47-4d5f-926e-e18fb5986dbb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.171067 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1408/2835] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:44.174843 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.184668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fab6-565c-406f-931b-92b73c6d6931 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.189144 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.189787 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.190164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.190439 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] PATH_INFO: `/v3/projects/c075cc7c78fd44529419b6837b88f24f/users/66535ad70a6c4b968bc8be17019cf1da/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.205055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c075cc7c78fd44529419b6837b88f24f, user_id=66535ad70a6c4b968bc8be17019cf1da, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.207689 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.220478 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1428/2836] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:44 2022] PUT /identity/v3/projects/c075cc7c78fd44529419b6837b88f24f/users/66535ad70a6c4b968bc8be17019cf1da/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:44.231908 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.232200 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.232487 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.260617 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] MFA Rules not processed for user `66535ad70a6c4b968bc8be17019cf1da`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:44.261846 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:44.325586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1409/2837] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:44.330063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.395600 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e5aed-915e-4fa5-9640-c1565ff78367 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.400259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.400872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.401156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.401430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.402138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.404041 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.409596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:44.434249 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1429/2838] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:44 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:44.438317 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.449947 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf701f26-cfa7-4aae-bbfd-1b69611d33fd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.503126 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:44.512477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.513098 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.513382 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.513657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.514356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.516295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.540263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1410/2839] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:44 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:44.541523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.552087 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8509-cbe4-4275-9c79-77861b49794a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.557333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.558240 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.558517 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.558790 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.559485 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.561062 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.571889 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1430/2840] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:44.576096 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.586568 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f1367-4df4-424c-8318-ff2c63267511 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.591331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.592002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.592619 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.592899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] PATH_INFO: `/v3/projects/933703e5fef142df81d602b5973d0d78/users/59bd29d25b014b3aa72ce4a6724f796d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.610153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=933703e5fef142df81d602b5973d0d78, user_id=59bd29d25b014b3aa72ce4a6724f796d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.613451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.632199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1411/2841] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:44 2022] PUT /identity/v3/projects/933703e5fef142df81d602b5973d0d78/users/59bd29d25b014b3aa72ce4a6724f796d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:44.633511 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.643705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153631-f39b-4f6b-9d4c-703b31a49ec2 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.708045 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.708727 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.709005 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.709278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.710038 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.711637 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad61026-2bd1-4a62-83ae-e587155ebc2b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.722512 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1431/2842] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:44.727261 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.738574 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80343-9896-4b93-92bd-0227b6e70c1d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.743753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.744415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.744694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.744970 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] PATH_INFO: `/v3/projects/933703e5fef142df81d602b5973d0d78/users/59bd29d25b014b3aa72ce4a6724f796d/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.747227 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] RBAC: Authorizing `identity:create_grant(project_id=933703e5fef142df81d602b5973d0d78, user_id=59bd29d25b014b3aa72ce4a6724f796d, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.749904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.760715 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1412/2843] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:44 2022] PUT /identity/v3/projects/933703e5fef142df81d602b5973d0d78/users/59bd29d25b014b3aa72ce4a6724f796d/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:44.771577 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.771872 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.772155 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.799931 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] MFA Rules not processed for user `59bd29d25b014b3aa72ce4a6724f796d`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:44.801150 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:44.875901 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1432/2844] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:44.887063 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:44.888872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:44.951296 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205acd-8725-4a71-831a-c7cf1142557a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:44.954958 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:44.955621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:44.955904 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:44.956215 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:44.967789 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:44.969338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b1ed2-c2fc-42a8-9756-e7932fb73116 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:44.979762 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1413/2845] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:45.007726 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:45.064833 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:45.069677 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e47935-40e6-40b8-b277-bf6b1a0cf095 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:45.076561 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:45.077239 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:45.077657 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:45.077969 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] PATH_INFO: `/v3/users/66535ad70a6c4b968bc8be17019cf1da` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:45.079715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] RBAC: Authorizing `identity:delete_user(user_id=66535ad70a6c4b968bc8be17019cf1da)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:45.082353 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:45.135801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66535ad70a6c4b968bc8be17019cf1da'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.136446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66535ad70a6c4b968bc8be17019cf1da'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.144324 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66535ad70a6c4b968bc8be17019cf1da'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.157152 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66535ad70a6c4b968bc8be17019cf1da'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.164604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66535ad70a6c4b968bc8be17019cf1da'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.170567 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1433/2846] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:45 2022] DELETE /identity/v3/users/66535ad70a6c4b968bc8be17019cf1da => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:45.175294 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:45.175581 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:45.175850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:45.195117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:45.196312 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:45.257857 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1414/2847] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:45.268447 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:45.269891 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:45.326612 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4cb13-5a54-403a-bfcb-c4370a2d339d admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:45.330228 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:45.330837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:45.331122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:45.331395 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:45.340776 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:45.342233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:45.351474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1434/2848] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:45.622967 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:45.624466 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:45.639978 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f011d5-5788-48e1-8f8a-8ba0cc13c064 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:45.644694 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:45.645342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:45.645626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:45.645944 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] PATH_INFO: `/v3/projects/c075cc7c78fd44529419b6837b88f24f` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:45.648255 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c075cc7c78fd44529419b6837b88f24f)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:45.650435 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:45.696213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:45.699454 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c075cc7c78fd44529419b6837b88f24f', 'request_id': 'req-7e2db230-212d-4322-b4d2-9749cb10fab4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.706702 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c075cc7c78fd44529419b6837b88f24f', 'request_id': 'req-7e2db230-212d-4322-b4d2-9749cb10fab4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.714686 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c075cc7c78fd44529419b6837b88f24f', 'request_id': 'req-7e2db230-212d-4322-b4d2-9749cb10fab4'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.725606 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] The token cache is being invalidate because project c075cc7c78fd44529419b6837b88f24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:45.725979 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.726609 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.728155 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1415/2849] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:45 2022] DELETE /identity/v3/projects/c075cc7c78fd44529419b6837b88f24f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:45.732085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:45.742423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d577d2-e5a2-4c32-8935-c5941fe5e1e9 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:45.812492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:45.813139 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:45.813418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:45.813693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] PATH_INFO: `/v3/users/59bd29d25b014b3aa72ce4a6724f796d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:45.815056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bd29d25b014b3aa72ce4a6724f796d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:45.817055 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:45.840077 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:45.868718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bd29d25b014b3aa72ce4a6724f796d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.869365 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bd29d25b014b3aa72ce4a6724f796d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.876355 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bd29d25b014b3aa72ce4a6724f796d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.889268 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bd29d25b014b3aa72ce4a6724f796d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.896688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-b9c3c76b-5586-4111-97ff-41a9690ef42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bd29d25b014b3aa72ce4a6724f796d'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:45.902827 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1435/2850] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:45 2022] DELETE /identity/v3/users/59bd29d25b014b3aa72ce4a6724f796d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:46.184595 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.186589 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:46.200731 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2db230-212d-4322-b4d2-9749cb10fab4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.262168 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.262985 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.263275 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.263572 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] PATH_INFO: `/v3/projects/933703e5fef142df81d602b5973d0d78` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.264727 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] RBAC: Authorizing `identity:delete_project(project_id=933703e5fef142df81d602b5973d0d78)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.266679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.314472 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933703e5fef142df81d602b5973d0d78', 'request_id': 'req-78c6e674-7581-443c-9630-9c22b262272c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:46.320592 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933703e5fef142df81d602b5973d0d78', 'request_id': 'req-78c6e674-7581-443c-9630-9c22b262272c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:46.327891 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933703e5fef142df81d602b5973d0d78', 'request_id': 'req-78c6e674-7581-443c-9630-9c22b262272c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:46.338963 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] The token cache is being invalidate because project 933703e5fef142df81d602b5973d0d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:46.339252 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:46.339885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-78c6e674-7581-443c-9630-9c22b262272c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:46.341406 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1416/2851] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:46 2022] DELETE /identity/v3/projects/933703e5fef142df81d602b5973d0d78 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:46.346426 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.346718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.346999 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.369125 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:46.370492 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:46.433521 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1436/2852] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:46.441164 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.441463 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.441755 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.462028 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:46.463235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:46.471462 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1417/2853] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:46.476140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.485991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83087-1b5d-4192-a2a3-06673af109cd None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.491168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.491824 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.492106 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.492584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.493362 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:46.493672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.495304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.507413 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1437/2854] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:46.511796 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.521708 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7555f-a1f3-4c16-adbb-f2bb675026e9 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.526501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.527111 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.527412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.527669 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.528325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.530282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.560277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1418/2855] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:46.564568 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.575202 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f34c98-a27d-4889-a55d-ec2285ff5a4a admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.638248 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.639110 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.639256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.639409 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.640081 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.642012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.666122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1438/2856] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:46.670248 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.681447 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bbb7-d308-44d3-8700-8e582ac6dcfb admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.686214 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.686828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.687132 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.687422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.688117 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.689701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.702076 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1419/2857] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 13:30:46.706328 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.718474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444d3e-0d6f-474a-a20c-bb554fb68ec3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.723356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.724080 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.724960 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.725307 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.726112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.727837 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e128d-e2d4-4862-9ad3-7e92f7111e87 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.739658 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1439/2858] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:46.743871 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:46.758726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac7bee-662b-4758-8008-6a089c190e14 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:46.764564 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:46.765489 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.765774 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.766085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] PATH_INFO: `/v3/projects/2ad7c526649d4a898e8da723502c6b73/users/d117a22abdde42d298cb27652673816a/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.783235 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad7c526649d4a898e8da723502c6b73, user_id=d117a22abdde42d298cb27652673816a, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:46.785938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:46.810834 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:46.815300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1420/2859] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:46 2022] PUT /identity/v3/projects/2ad7c526649d4a898e8da723502c6b73/users/d117a22abdde42d298cb27652673816a/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:46.827410 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:46.827547 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:46.827826 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:46.836672 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:46.863003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] MFA Rules not processed for user `d117a22abdde42d298cb27652673816a`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:46.864469 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:46.932723 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1440/2860] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:46.937304 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.006119 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b45f7-5ce5-4250-8cc9-7003c524fa98 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.011158 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.011909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.012193 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.012468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.013144 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.015084 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.048379 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1421/2861] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:46 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:47.049366 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.060596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f993c2e-1900-4235-8045-c9fe9ad87b19 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.127923 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.128450 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.128732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.129008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.129688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.132423 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.156153 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1441/2862] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:47 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:47.160120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.171459 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d14d5e-2158-4a4a-84f2-f56f28a17586 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.176688 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.177344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.177628 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.177938 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.178640 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.180269 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.191788 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1422/2863] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:47.196581 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.207480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407161-e12f-4415-81c3-d85a0c8a40a7 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.213036 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.213715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.214028 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.214304 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] PATH_INFO: `/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0/users/cb1e576f429e4474ad988f418b353a5c/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.229319 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c7b79f19924223a8a0f2a3eebc65f0, user_id=cb1e576f429e4474ad988f418b353a5c, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.231988 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.247138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1442/2864] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:47 2022] PUT /identity/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0/users/cb1e576f429e4474ad988f418b353a5c/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:47.251050 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.261842 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce3d18-3751-49a1-92aa-716fcc145def admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.321370 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.322031 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.322319 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.322596 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.323300 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.324893 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13229929-fc71-488b-aa6f-f90d2c1c50b0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.335306 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1423/2865] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 13:30:47.339416 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.351751 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6cbcc-8c2c-4053-9690-80e8974d4be8 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.358736 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.359456 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.359738 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.360016 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] PATH_INFO: `/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0/users/cb1e576f429e4474ad988f418b353a5c/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.362056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c7b79f19924223a8a0f2a3eebc65f0, user_id=cb1e576f429e4474ad988f418b353a5c, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.365233 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.376169 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1443/2866] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:47 2022] PUT /identity/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0/users/cb1e576f429e4474ad988f418b353a5c/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:47.387423 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.387740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.388016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.415872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] MFA Rules not processed for user `cb1e576f429e4474ad988f418b353a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:47.417093 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:47.477130 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:47.486943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1424/2867] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:47.499477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.501278 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:47.556151 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:47.563831 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3315-a45b-4881-8b60-5d5e21483dbb admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.567735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.569250 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.569534 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.569839 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.579291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.580732 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.592116 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1444/2868] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:47.680615 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:47.690634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5875a8-6797-488f-9166-d0a8764ec564 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:47.694444 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:47.695097 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:47.695378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:47.695726 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:47.754768 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:47.756385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a9ff1-8185-45a7-a99e-8ffe706c3008 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:47.765602 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1425/2869] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:52.502112 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:52.567008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d749-f3fe-49d5-b683-80746cc52fb9 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:52.572136 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:52.572775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:52.573056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:52.573333 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] PATH_INFO: `/v3/users/d117a22abdde42d298cb27652673816a` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:52.574513 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d117a22abdde42d298cb27652673816a)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:52.576508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:52.624741 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd117a22abdde42d298cb27652673816a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:52.625331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd117a22abdde42d298cb27652673816a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:52.633995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd117a22abdde42d298cb27652673816a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:52.647644 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd117a22abdde42d298cb27652673816a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:52.654531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd117a22abdde42d298cb27652673816a'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:52.660430 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1445/2870] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:52 2022] DELETE /identity/v3/users/d117a22abdde42d298cb27652673816a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:52.664976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:52.665451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:52.665566 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:52.684817 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:52.686039 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:52.744295 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1426/2871] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 13:30:52.754075 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:52.755528 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:52.816798 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b2307-41e4-4280-be4e-159669967eb3 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:52.820472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:52.821093 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:52.821375 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:52.821651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:52.830725 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:52.832611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:52.843185 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1446/2872] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 13:30:53.127015 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:53.128464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:53.141689 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7ab49-1e6e-4dd6-a74a-91aa349c207d None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:53.157520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:53.158292 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.158497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.158773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] PATH_INFO: `/v3/projects/2ad7c526649d4a898e8da723502c6b73` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.159929 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad7c526649d4a898e8da723502c6b73)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:53.161883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:53.212127 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad7c526649d4a898e8da723502c6b73', 'request_id': 'req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.219221 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad7c526649d4a898e8da723502c6b73', 'request_id': 'req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.227398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad7c526649d4a898e8da723502c6b73', 'request_id': 'req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.238210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] The token cache is being invalidate because project 2ad7c526649d4a898e8da723502c6b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:53.238503 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.239246 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.241017 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1427/2873] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:53 2022] DELETE /identity/v3/projects/2ad7c526649d4a898e8da723502c6b73 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:53.244991 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:53.254899 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd13a4-7908-4f0e-b835-76f86413bf38 service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:53.262470 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:53.312693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:53.313330 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.313606 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.313904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] PATH_INFO: `/v3/users/cb1e576f429e4474ad988f418b353a5c` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.315073 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1e576f429e4474ad988f418b353a5c)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:53.316970 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:53.363896 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1e576f429e4474ad988f418b353a5c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.364546 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1e576f429e4474ad988f418b353a5c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.372774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1e576f429e4474ad988f418b353a5c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.389313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1e576f429e4474ad988f418b353a5c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.396668 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-a9e1f641-6ee4-4ba1-a6dd-087aebbb0d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1e576f429e4474ad988f418b353a5c'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.403813 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1447/2874] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:30:53 2022] DELETE /identity/v3/users/cb1e576f429e4474ad988f418b353a5c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:53.684500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:53.684500 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:53.699461 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa9d33-7d07-4fbd-8e06-d4ca9d48658d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:53.717468 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:53.764665 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:53.765219 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.765501 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.765808 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] PATH_INFO: `/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.767016 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c7b79f19924223a8a0f2a3eebc65f0)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:53.768962 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:53.814847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c7b79f19924223a8a0f2a3eebc65f0', 'request_id': 'req-e1461156-59e6-4ddc-ae38-fba98622b14a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.821477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c7b79f19924223a8a0f2a3eebc65f0', 'request_id': 'req-e1461156-59e6-4ddc-ae38-fba98622b14a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.829342 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c7b79f19924223a8a0f2a3eebc65f0', 'request_id': 'req-e1461156-59e6-4ddc-ae38-fba98622b14a'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.841507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] The token cache is being invalidate because project 58c7b79f19924223a8a0f2a3eebc65f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91820) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:30:53.841800 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.842486 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-e1461156-59e6-4ddc-ae38-fba98622b14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:30:53.844002 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1428/2875] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:30:53 2022] DELETE /identity/v3/projects/58c7b79f19924223a8a0f2a3eebc65f0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:53.848979 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.849446 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.849535 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.873197 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:53.874530 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:53.934003 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1448/2876] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:53.943210 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.943498 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.943773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.964605 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:53.966010 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:53.974153 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1429/2877] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:53.978629 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:53.988531 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698fbe7-47eb-434f-b5e6-ac8eeab634ae None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:53.993579 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:53.994229 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:53.994519 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:53.994795 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] PATH_INFO: `/v3/domains` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:53.995488 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91819) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:30:53.995794 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:53.997625 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.011084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1449/2878] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:30:54.015013 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.025198 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3ca70-8e5d-46e6-ab12-43b12f85b0fd None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.029793 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.030422 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.030700 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.030971 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.031632 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.033542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.061863 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1430/2879] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:30:54.066208 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.077256 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff2aa7-ee2e-4469-bb61-125d1ce1b335 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.091807 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:54.139898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.140523 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.140806 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.141084 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.141759 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.143702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.164288 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1450/2880] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:54.168464 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.179391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17d344-b7d3-455e-b307-e90fac902980 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.184162 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.184786 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.185066 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.185341 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.186075 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.187676 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.201545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1431/2881] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:54.203114 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.213491 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341ef0-5651-4efa-b017-0a08933e278c admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.218481 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.219097 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.219379 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.219651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.220344 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.221957 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de40dd-e222-40ba-8baa-8b3fb157cb55 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.233440 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1451/2882] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:54.238398 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.250451 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc7726-e6fe-40c7-9154-68fb375fd21c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.255060 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.255720 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.255999 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.256278 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] PATH_INFO: `/v3/projects/89426baa12844371a615261e743846b0/users/a0d973991fec4a89a6475ed10b72ced2/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.272750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89426baa12844371a615261e743846b0, user_id=a0d973991fec4a89a6475ed10b72ced2, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.275364 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.289092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1432/2883] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:54 2022] PUT /identity/v3/projects/89426baa12844371a615261e743846b0/users/a0d973991fec4a89a6475ed10b72ced2/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:54.299480 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573534db-5674-4789-ac62-e61e2273896e None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.299771 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573534db-5674-4789-ac62-e61e2273896e None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.300054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573534db-5674-4789-ac62-e61e2273896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.328874 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-573534db-5674-4789-ac62-e61e2273896e None None] MFA Rules not processed for user `a0d973991fec4a89a6475ed10b72ced2`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:54.330121 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-573534db-5674-4789-ac62-e61e2273896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:54.393978 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1452/2884] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:54.398550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.446107 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:54.465325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0bf19-7087-43fd-8471-941c1be1e003 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.470230 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.470850 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.471135 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.471412 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.472088 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.474027 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.506277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1433/2885] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:54.507618 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573534db-5674-4789-ac62-e61e2273896e None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.518888 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573534db-5674-4789-ac62-e61e2273896e None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.577472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.578175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.578460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.578739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.579427 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.581502 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.613475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1453/2886] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:54.617883 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.628378 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f48e4-26f3-4770-a178-c30760e36761 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.632964 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.633571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.633882 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.634171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.634909 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.636522 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.647847 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1434/2887] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:54.651863 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.662471 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbfbc-9232-44bd-9e25-2901ad427160 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.668056 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.668685 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.668968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.669242 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.670138 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.671743 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e382c9-cf8c-4139-bf72-2b427e57ae1e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.684633 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1454/2888] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:54.689045 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.699740 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a54cc-e2d7-40f9-bfe0-6bbe523bd829 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.704373 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.705065 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.705344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.705621 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] PATH_INFO: `/v3/projects/646d5d5fecad41ad9abaee8cb69ef0aa/users/5e25fae361a94b0283a03ddaa30f50a7/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.721723 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646d5d5fecad41ad9abaee8cb69ef0aa, user_id=5e25fae361a94b0283a03ddaa30f50a7, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.724437 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.738570 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1435/2889] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:54 2022] PUT /identity/v3/projects/646d5d5fecad41ad9abaee8cb69ef0aa/users/5e25fae361a94b0283a03ddaa30f50a7/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:30:54.749273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.749563 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.749855 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.757981 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:54.779778 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] MFA Rules not processed for user `5e25fae361a94b0283a03ddaa30f50a7`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:54.781008 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:54.848177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1455/2890] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:54.853184 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.916229 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31683f-1f71-4f91-a284-2eb5e610dce0 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:54.920976 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:54.921599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:54.921913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:54.922192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:54.922859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:54.924766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:54.955085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1436/2891] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:54.960156 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:54.974814 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c0dcd-f2af-4d5f-9469-ea87b3347428 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.033477 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.034141 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.034421 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.034693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.035359 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.037270 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.060508 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1456/2892] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:30:54 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:30:55.064634 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:55.074875 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624dc8-7819-44ef-bb7a-236b37c79b0a admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.079670 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.080338 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.080550 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.080818 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.081625 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.083391 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.095021 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1437/2893] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:55.099309 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:55.109690 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417468-10d3-46d6-96f7-2082273fa0c4 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.114350 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.115018 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.115305 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.115583 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] PATH_INFO: `/v3/projects/2ba14cc3b9774386994a9d7f85a532f4/users/f8d1ebeee2154937a74d368a1b5f456b/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.135472 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba14cc3b9774386994a9d7f85a532f4, user_id=f8d1ebeee2154937a74d368a1b5f456b, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.138154 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.151394 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1457/2894] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:55 2022] PUT /identity/v3/projects/2ba14cc3b9774386994a9d7f85a532f4/users/f8d1ebeee2154937a74d368a1b5f456b/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:55.155460 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:55.166121 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f54d40-ea95-49da-a41c-36f240172a7e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.224639 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.226169 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.226452 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.226878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.227753 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.229527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b245ba7-f79b-493a-9d64-9f5cae94ee35 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.239516 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1438/2895] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:30:55.248204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:55.260538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be23a8e-4e54-4d89-a1e7-68d8b48f0e98 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.267801 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.268418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.268760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.269088 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] PATH_INFO: `/v3/projects/2ba14cc3b9774386994a9d7f85a532f4/users/f8d1ebeee2154937a74d368a1b5f456b/roles/9419bac9c2624887a6ae8f158b33c48d` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.275489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba14cc3b9774386994a9d7f85a532f4, user_id=f8d1ebeee2154937a74d368a1b5f456b, role_id=9419bac9c2624887a6ae8f158b33c48d)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.278474 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.294196 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1458/2896] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:30:55 2022] PUT /identity/v3/projects/2ba14cc3b9774386994a9d7f85a532f4/users/f8d1ebeee2154937a74d368a1b5f456b/roles/9419bac9c2624887a6ae8f158b33c48d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:30:55.314922 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.315243 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.315488 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.329555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:55.352276 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] MFA Rules not processed for user `f8d1ebeee2154937a74d368a1b5f456b`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:30:55.353411 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:55.422457 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1439/2897] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:30:55.435459 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:55.437175 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:30:55.493436 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:30:55.499990 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae274230-f19f-47be-8c88-77a5890c71f6 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:55.503968 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:55.504635 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:55.504904 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:55.505178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:55.563377 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:55.564955 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:55.574504 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1459/2898] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:57.091171 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:57.100703 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82324a-e78a-4f12-b22a-bd869345d352 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:57.104173 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:57.104779 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:57.105081 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:57.105351 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:57.162340 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:57.163737 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:57.172517 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1440/2899] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:30:57.706950 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:30:57.716049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704460-9024-4f88-bf31-22836b8c89d3 service cinder] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:30:57.719664 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:30:57.720273 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:30:57.720559 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:30:57.720834 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:30:57.730348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:30:57.731770 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b0ac5-f447-4863-8470-43244d7c9f9d service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:30:57.741054 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1460/2900] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:31:01.586273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:01.649520 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a820f5-855b-49eb-bff5-cf6fc50b1690 service cinder] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:01.654105 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:01.654749 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:01.655035 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:01.655314 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] PATH_INFO: `/v3/users/a0d973991fec4a89a6475ed10b72ced2` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:01.656478 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d973991fec4a89a6475ed10b72ced2)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:01.658438 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:01.694876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:01.712537 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d973991fec4a89a6475ed10b72ced2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:01.719691 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d973991fec4a89a6475ed10b72ced2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:01.720418 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d973991fec4a89a6475ed10b72ced2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:01.732829 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d973991fec4a89a6475ed10b72ced2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:01.733496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d973991fec4a89a6475ed10b72ced2'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:01.749176 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1441/2901] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:31:01 2022] DELETE /identity/v3/users/a0d973991fec4a89a6475ed10b72ced2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:01.753810 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:01.754338 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:01.754442 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:01.776173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:01.777858 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:01.845984 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1461/2902] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:01.856470 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:01.858018 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:01.921273 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909140e-083a-4dec-b9bf-9abd413166ad admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:01.925177 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:01.925931 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:01.926265 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:01.926545 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:01.937865 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:01.939202 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:01.949542 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1442/2903] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:31:02.237096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:02.238575 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:02.251760 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07cca3-2626-4171-b0af-229e2895d68f None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:02.257212 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:02.258012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:02.258385 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:02.259178 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] PATH_INFO: `/v3/projects/89426baa12844371a615261e743846b0` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:02.261014 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=89426baa12844371a615261e743846b0)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:02.262971 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:02.314611 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89426baa12844371a615261e743846b0', 'request_id': 'req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.320247 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89426baa12844371a615261e743846b0', 'request_id': 'req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.327058 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89426baa12844371a615261e743846b0', 'request_id': 'req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.340773 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] The token cache is being invalidate because project 89426baa12844371a615261e743846b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:31:02.341063 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.341892 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.343418 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1462/2904] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:31:02 2022] DELETE /identity/v3/projects/89426baa12844371a615261e743846b0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:31:02.347323 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:02.357696 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d55a41-dfb0-4250-b21b-8f47d3812089 service neutron] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:02.412679 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:02.413309 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:02.414240 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:02.414532 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] PATH_INFO: `/v3/users/5e25fae361a94b0283a03ddaa30f50a7` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:02.415697 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e25fae361a94b0283a03ddaa30f50a7)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:02.417608 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:02.462626 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e25fae361a94b0283a03ddaa30f50a7'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.466885 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:02.470709 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e25fae361a94b0283a03ddaa30f50a7'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.471555 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e25fae361a94b0283a03ddaa30f50a7'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.484577 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e25fae361a94b0283a03ddaa30f50a7'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.485163 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e25fae361a94b0283a03ddaa30f50a7'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.497610 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1443/2905] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:31:02 2022] DELETE /identity/v3/users/5e25fae361a94b0283a03ddaa30f50a7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:02.771915 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:02.774371 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:02.787705 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5092f9-faa9-4a8d-be14-a6a3e51b18ee admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:02.797538 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:02.848299 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:02.848946 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:02.849231 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:02.849533 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] PATH_INFO: `/v3/projects/646d5d5fecad41ad9abaee8cb69ef0aa` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:02.850702 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646d5d5fecad41ad9abaee8cb69ef0aa)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:02.852626 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:02.898854 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646d5d5fecad41ad9abaee8cb69ef0aa', 'request_id': 'req-e5ea31da-ae71-496f-acf7-2a38190ab592'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.904012 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646d5d5fecad41ad9abaee8cb69ef0aa', 'request_id': 'req-e5ea31da-ae71-496f-acf7-2a38190ab592'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.911122 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646d5d5fecad41ad9abaee8cb69ef0aa', 'request_id': 'req-e5ea31da-ae71-496f-acf7-2a38190ab592'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.922253 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] The token cache is being invalidate because project 646d5d5fecad41ad9abaee8cb69ef0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:31:02.922541 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.923164 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:02.924653 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1463/2906] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:31:02 2022] DELETE /identity/v3/projects/646d5d5fecad41ad9abaee8cb69ef0aa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:02.928795 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:02.930750 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:02.943571 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefe9f7-7f32-4944-b7a4-ee331a125e17 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.000477 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.001234 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.001525 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.001804 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] PATH_INFO: `/v3/users/f8d1ebeee2154937a74d368a1b5f456b` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.003085 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d1ebeee2154937a74d368a1b5f456b)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.005007 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.053375 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d1ebeee2154937a74d368a1b5f456b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.060666 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d1ebeee2154937a74d368a1b5f456b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.061325 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d1ebeee2154937a74d368a1b5f456b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.073146 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d1ebeee2154937a74d368a1b5f456b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.073777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-26694977-7f61-4916-9aa6-0cc15d1a56e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d1ebeee2154937a74d368a1b5f456b'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.087091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1444/2907] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:31:02 2022] DELETE /identity/v3/users/f8d1ebeee2154937a74d368a1b5f456b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:03.361118 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.362553 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:03.377176 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea31da-ae71-496f-acf7-2a38190ab592 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.434852 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.435499 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.435781 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.436060 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] PATH_INFO: `/v3/projects/2ba14cc3b9774386994a9d7f85a532f4` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.437204 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba14cc3b9774386994a9d7f85a532f4)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.439170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.483803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:03.487995 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba14cc3b9774386994a9d7f85a532f4', 'request_id': 'req-adac22a9-ae05-4796-b8c5-9141f9be0c8e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.494774 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba14cc3b9774386994a9d7f85a532f4', 'request_id': 'req-adac22a9-ae05-4796-b8c5-9141f9be0c8e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.502261 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba14cc3b9774386994a9d7f85a532f4', 'request_id': 'req-adac22a9-ae05-4796-b8c5-9141f9be0c8e'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.513461 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] The token cache is being invalidate because project 2ba14cc3b9774386994a9d7f85a532f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:31:03.513748 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.514401 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-adac22a9-ae05-4796-b8c5-9141f9be0c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:03.515890 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1464/2908] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:31:03 2022] DELETE /identity/v3/projects/2ba14cc3b9774386994a9d7f85a532f4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:03.520701 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.520993 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.521263 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.540769 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:03.541913 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:03.584657 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:03.601747 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1445/2909] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:03.609422 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.609715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.610031 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.630515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:03.631722 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:03.640173 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1465/2910] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:03.644356 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.654872 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422700b7-65ca-4832-bb0c-4a40355f8901 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.659359 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.659957 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.660236 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.660508 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] PATH_INFO: `/v3/domains` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.661199 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91820) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 13:31:03.661499 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.663120 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.674307 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1446/2911] 10.222.0.66 () {58 vars in 1153 bytes} [Fri Jul 29 13:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 13:31:03.678348 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.688170 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce5b68-910b-45da-9523-3e1c2be47325 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.692803 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.693402 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.693688 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.693993 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.694647 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.697165 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.723356 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1466/2912] 10.222.0.66 () {60 vars in 1153 bytes} [Fri Jul 29 13:31:03 2022] POST /identity/v3/projects => generated 395 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 13:31:03.727415 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.739824 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f8263-c4bc-4c19-8b03-85fffb0a954e admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.798115 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.798655 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.798937 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.799213 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] PATH_INFO: `/v3/users` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.799866 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.801773 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.822998 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1447/2913] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 29 13:31:03 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 13:31:03.827140 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.837687 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568a67-3566-4d93-93ec-fd57acbb1a69 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.842420 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.843029 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.843313 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.843614 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.844310 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.845937 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.856262 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1467/2914] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:31:03.860205 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.871101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b2b31-a051-4a24-9859-7654b284899c admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.875972 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.876599 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.876878 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.877152 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] PATH_INFO: `/v3/roles` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.877899 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.879497 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440f1b-b355-4229-86ce-9e97f3111649 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.890586 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1448/2915] 10.222.0.66 () {58 vars in 1122 bytes} [Fri Jul 29 13:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 13:31:03.894762 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:03.904693 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f15f93-ee0e-4703-973c-7fd298bf7d99 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:03.909291 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:03.909985 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.910271 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.910551 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] PATH_INFO: `/v3/projects/f018f2ce263a4e39affe834b5b92a710/users/0f2cd1e6a3584c42b46671cca14b419e/roles/2c1eb45dfce946f99346c70e0f0906ae` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.924869 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f018f2ce263a4e39affe834b5b92a710, user_id=0f2cd1e6a3584c42b46671cca14b419e, role_id=2c1eb45dfce946f99346c70e0f0906ae)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:03.927572 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:03.940549 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1468/2916] 10.222.0.66 () {60 vars in 1483 bytes} [Fri Jul 29 13:31:03 2022] PUT /identity/v3/projects/f018f2ce263a4e39affe834b5b92a710/users/0f2cd1e6a3584c42b46671cca14b419e/roles/2c1eb45dfce946f99346c70e0f0906ae => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 13:31:03.952192 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:03.952682 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:03.952777 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:03.980448 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] MFA Rules not processed for user `0f2cd1e6a3584c42b46671cca14b419e`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:03.981645 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:04.046371 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1449/2917] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:04.057883 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:04.059596 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:04.123467 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e377a-59f2-49ce-acb3-05fe24056a7f admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:04.127085 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1a7571787a4415fb254a652062e50f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:04.127718 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:04.127982 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:04.128251 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:04.136655 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:04.138096 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41eec98-a9cd-4998-aaeb-bd9af69891ed service cinder] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:04.146739 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1469/2918] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:31:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:31:14.043108 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:14.142825 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:14.304189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:14.366151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a9a74-f557-4416-b15a-3761d9f9d68e None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:14.370651 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:14.371277 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:14.371816 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:14.372101 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] PATH_INFO: `/v3/users/0f2cd1e6a3584c42b46671cca14b419e` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:14.373280 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2cd1e6a3584c42b46671cca14b419e)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:14.375223 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:14.420680 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2cd1e6a3584c42b46671cca14b419e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:14.427509 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2cd1e6a3584c42b46671cca14b419e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:14.428203 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2cd1e6a3584c42b46671cca14b419e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:14.442394 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2cd1e6a3584c42b46671cca14b419e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:14.443091 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2cd1e6a3584c42b46671cca14b419e'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:14.458367 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1450/2919] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:31:14 2022] DELETE /identity/v3/users/0f2cd1e6a3584c42b46671cca14b419e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:14.464489 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] REQUEST_METHOD: `POST` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:14.464775 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:14.465049 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:14.484515 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.auth.core [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] MFA Rules not processed for user `1eea4fac81544f2bac9c90f3a5feb407`. Rule list: `[]` (Enabled: `True`). {{(pid=91819) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:14.485715 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:14.544733 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1470/2920] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:14.554496 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:14.555828 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:14.612331 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2badf-b6e3-4503-9d36-bcedf6dfc26d admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:14.616006 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:14.617114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:14.617114 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:14.618161 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:14.626839 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:14.628262 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669074-b056-442a-b969-ec985cb782b5 service neutron] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:14.638092 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1451/2921] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:31:14.919246 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:14.920740 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:14.934669 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50a95-9c47-4ea9-a50f-3574e02c4fe3 None None] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:14.940047 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:14.940861 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:14.941301 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:14.941497 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] PATH_INFO: `/v3/projects/f018f2ce263a4e39affe834b5b92a710` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:14.946023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f018f2ce263a4e39affe834b5b92a710)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:14.946023 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:15.003604 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f018f2ce263a4e39affe834b5b92a710', 'request_id': 'req-bea50940-13c9-4774-a5fa-6c1a54956ba5'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:15.010306 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f018f2ce263a4e39affe834b5b92a710', 'request_id': 'req-bea50940-13c9-4774-a5fa-6c1a54956ba5'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:15.018065 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f018f2ce263a4e39affe834b5b92a710', 'request_id': 'req-bea50940-13c9-4774-a5fa-6c1a54956ba5'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:15.028976 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] The token cache is being invalidate because project f018f2ce263a4e39affe834b5b92a710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:31:15.029259 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:15.029898 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:31:15.031387 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1471/2922] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:31:14 2022] DELETE /identity/v3/projects/f018f2ce263a4e39affe834b5b92a710 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:31:24.634020 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:25.025994 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:40.352151 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] REQUEST_METHOD: `POST` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:40.352638 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:40.352725 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:40.372133 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.auth.core [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] MFA Rules not processed for user `96422547f7fb45a7a0920e520d66870c`. Rule list: `[]` (Enabled: `True`). {{(pid=91820) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 13:31:40.373377 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:40.427507 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1452/2923] 10.222.0.66 () {58 vars in 958 bytes} [Fri Jul 29 13:31:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 13:31:50.420943 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=25 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:31:52.285331 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:31:52.287013 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:52.368651 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50940-13c9-4774-a5fa-6c1a54956ba5 admin admin] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:31:52.372558 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c22269afbdf541d4bedbef2cf8503521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:31:52.373177 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] REQUEST_METHOD: `GET` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:31:52.373460 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:31:52.373735 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:31:52.375520 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.fernet_utils [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91819) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:31:52.385127 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:31:52.386584 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:31:52.395168 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1472/2924] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:32:02.390966 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=30 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:32:38.861385 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:32:38.862901 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:32:38.947053 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2c82a-f555-425e-992f-872742af8f37 None None] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:32:38.951884 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:32:38.952527 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:32:38.952809 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:32:38.953083 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] PATH_INFO: `/v3/users/96422547f7fb45a7a0920e520d66870c` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:32:38.954282 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96422547f7fb45a7a0920e520d66870c)` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:32:38.956201 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:32:39.001876 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96422547f7fb45a7a0920e520d66870c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.009505 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96422547f7fb45a7a0920e520d66870c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.010233 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96422547f7fb45a7a0920e520d66870c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.021928 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96422547f7fb45a7a0920e520d66870c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.022561 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.notifications [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96422547f7fb45a7a0920e520d66870c'} {{(pid=91820) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.034826 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1453/2925] 10.222.0.66 () {58 vars in 1224 bytes} [Fri Jul 29 13:32:38 2022] DELETE /identity/v3/users/96422547f7fb45a7a0920e520d66870c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:32:39.203666 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] Authenticating user token {{(pid=91819) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:32:39.220004 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b898ce7-d4c8-43a1-b021-9ccb3ab422ae service neutron] Validating token access rules against request {{(pid=91819) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:32:39.276205 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eea4fac81544f2bac9c90f3a5feb407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e5a5cdc1deb42199be2e349eba98664', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91819) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:32:39.276859 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:32:39.277142 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:32:39.277417 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] PATH_INFO: `/v3/projects/9a86813dfa9c4d0d8c0217d29d1318f5` {{(pid=91819) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:32:39.278593 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a86813dfa9c4d0d8c0217d29d1318f5)` {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:32:39.280475 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] RBAC: Authorization granted {{(pid=91819) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:32:39.325149 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a86813dfa9c4d0d8c0217d29d1318f5', 'request_id': 'req-497659c0-ff9d-4342-ad17-008ddb2bc0ba'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.330527 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a86813dfa9c4d0d8c0217d29d1318f5', 'request_id': 'req-497659c0-ff9d-4342-ad17-008ddb2bc0ba'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.338390 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a86813dfa9c4d0d8c0217d29d1318f5', 'request_id': 'req-497659c0-ff9d-4342-ad17-008ddb2bc0ba'} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.349098 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] The token cache is being invalidate because project 9a86813dfa9c4d0d8c0217d29d1318f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91819) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 13:32:39.349388 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.350035 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG keystone.notifications [None req-497659c0-ff9d-4342-ad17-008ddb2bc0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91819) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 13:32:39.352050 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: [pid: 91819|app: 0|req: 1473/2926] 10.222.0.66 () {58 vars in 1233 bytes} [Fri Jul 29 13:32:39 2022] DELETE /identity/v3/projects/9a86813dfa9c4d0d8c0217d29d1318f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 13:32:49.031344 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:32:49.346201 dv-cinder-819149-16-iscsi devstack@keystone.service[91819]: DEBUG dbcounter [-] [91819] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91819) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 13:34:50.943772 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Authenticating user token {{(pid=91820) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 13:34:50.945189 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:34:51.024291 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185460-5d42-49cf-b60a-ad83c98f5fc4 admin admin] Validating token access rules against request {{(pid=91820) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 13:34:51.083859 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b74de7e13793482882a0097940e339c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2dfaace737a9483dade84c96710c7e54', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91820) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 13:34:51.084480 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] REQUEST_METHOD: `GET` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 13:34:51.084766 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] SCRIPT_NAME: `/identity` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 13:34:51.085044 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91820) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 13:34:51.086836 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.fernet_utils [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91820) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 13:34:51.099785 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 13:34:51.101431 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02343a7-838d-4419-ae1b-2044559ce1f0 service placement] RBAC: Authorization granted {{(pid=91820) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 13:34:51.111543 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: [pid: 91820|app: 0|req: 1454/2927] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 29 13:34:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 13:35:01.107207 dv-cinder-819149-16-iscsi devstack@keystone.service[91820]: DEBUG dbcounter [-] [91820] Writing DB stats keystone:SELECT=61 {{(pid=91820) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}